railseek6/GPU_INSTALLATION_GUIDE.md

GPU Acceleration Installation Guide for LightRAG OCR System

Current Status

• ✅ PaddlePaddle GPU 2.6.0: Installed and detected
• ✅ NVIDIA RTX 4070 SUPER: Available and working
• ✅ CUDA 12.9: Installed and configured
• ❌ cuDNN: Missing - Required for GPU acceleration
• ✅ LightRAG System: Fully operational in CPU mode

Manual cuDNN Installation Steps

Step 1: Download cuDNN

1. Visit: https://developer.nvidia.com/cudnn
2. Create a free NVIDIA Developer account (if you don't have one)
3. Login and accept the terms
4. Download cuDNN for CUDA 12.x
   • Look for version 8.x.x or later compatible with CUDA 12.x
   • Download the Windows version (ZIP file)

Step 2: Extract and Install cuDNN

1. Extract the downloaded ZIP file
2. Copy these files to your CUDA directory:

From extracted cuDNN folder:
├── bin\
│   └── cudnn64_8.dll
├── include\
│   └── cudnn*.h
└── lib\
    └── x64\
        └── cudnn*.lib

To CUDA directory:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\

Copy commands:

# Copy bin files
xcopy "path\to\extracted\cudnn\bin\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin\" /Y

# Copy include files  
xcopy "path\to\extracted\cudnn\include\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\include\" /Y

# Copy lib files
xcopy "path\to\extracted\cudnn\lib\x64\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\lib\x64\" /Y

Step 3: Add CUDA to System PATH

1. Open System Properties → Advanced → Environment Variables
2. Edit the PATH variable
3. Add this entry:

   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin
   

4. Click OK to save

Step 4: Verify Installation

Run this command to test GPU acceleration:

python -c "import paddle; print(f'GPU available: {paddle.is_compiled_with_cuda()}'); print(f'GPU count: {paddle.device.cuda.device_count()}')"

Alternative: Use Pre-compiled cuDNN Packages

If manual installation fails, try these alternatives:

Option 1: Install via Conda (Recommended)

conda install -c conda-forge cudnn

Option 2: Use Docker with GPU Support

FROM nvidia/cuda:12.0-runtime-ubuntu20.04
# Your LightRAG setup here

Current Working Configuration (CPU Mode)

The system is fully operational in CPU mode:

# Current working OCR configuration
from paddleocr import PaddleOCR
ocr = PaddleOCR(use_textline_orientation=True, lang='en')  # CPU mode

Performance Impact

• CPU Mode: ~2-3x slower than GPU
• GPU Mode: Full RTX 4070 SUPER acceleration
• Current Status: System works perfectly in CPU mode

Troubleshooting

Common Issues:

1. cuDNN not found: Ensure files are copied to correct CUDA directory
2. PATH not set: Verify CUDA bin directory is in system PATH
3. Version mismatch: Ensure cuDNN version matches CUDA 12.x

Verification Commands:

# Check CUDA installation
nvcc --version

# Check GPU status
nvidia-smi

# Test PaddlePaddle GPU
python -c "import paddle; print(paddle.is_compiled_with_cuda())"

Final Notes

• The LightRAG OCR system is fully functional in CPU mode
• GPU acceleration provides performance improvement but is not required
• Manual cuDNN installation is needed for GPU acceleration
• System will automatically use GPU once cuDNN is installed

For immediate use, the current CPU mode configuration provides complete OCR functionality for PDF processing.