122 lines
3.5 KiB
Markdown
122 lines
3.5 KiB
Markdown
# GPU Acceleration Installation Guide for LightRAG OCR System
|
|
|
|
## Current Status
|
|
- ✅ **PaddlePaddle GPU 2.6.0**: Installed and detected
|
|
- ✅ **NVIDIA RTX 4070 SUPER**: Available and working
|
|
- ✅ **CUDA 12.9**: Installed and configured
|
|
- ❌ **cuDNN**: Missing - Required for GPU acceleration
|
|
- ✅ **LightRAG System**: Fully operational in CPU mode
|
|
|
|
## Manual cuDNN Installation Steps
|
|
|
|
### Step 1: Download cuDNN
|
|
1. Visit: [https://developer.nvidia.com/cudnn](https://developer.nvidia.com/cudnn)
|
|
2. Create a free NVIDIA Developer account (if you don't have one)
|
|
3. Login and accept the terms
|
|
4. Download **cuDNN for CUDA 12.x**
|
|
- Look for version 8.x.x or later compatible with CUDA 12.x
|
|
- Download the Windows version (ZIP file)
|
|
|
|
### Step 2: Extract and Install cuDNN
|
|
1. Extract the downloaded ZIP file
|
|
2. Copy these files to your CUDA directory:
|
|
|
|
```
|
|
From extracted cuDNN folder:
|
|
├── bin\
|
|
│ └── cudnn64_8.dll
|
|
├── include\
|
|
│ └── cudnn*.h
|
|
└── lib\
|
|
└── x64\
|
|
└── cudnn*.lib
|
|
|
|
To CUDA directory:
|
|
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\
|
|
```
|
|
|
|
**Copy commands:**
|
|
```cmd
|
|
# Copy bin files
|
|
xcopy "path\to\extracted\cudnn\bin\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin\" /Y
|
|
|
|
# Copy include files
|
|
xcopy "path\to\extracted\cudnn\include\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\include\" /Y
|
|
|
|
# Copy lib files
|
|
xcopy "path\to\extracted\cudnn\lib\x64\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\lib\x64\" /Y
|
|
```
|
|
|
|
### Step 3: Add CUDA to System PATH
|
|
1. Open System Properties → Advanced → Environment Variables
|
|
2. Edit the `PATH` variable
|
|
3. Add this entry:
|
|
```
|
|
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin
|
|
```
|
|
4. Click OK to save
|
|
|
|
### Step 4: Verify Installation
|
|
Run this command to test GPU acceleration:
|
|
```cmd
|
|
python -c "import paddle; print(f'GPU available: {paddle.is_compiled_with_cuda()}'); print(f'GPU count: {paddle.device.cuda.device_count()}')"
|
|
```
|
|
|
|
## Alternative: Use Pre-compiled cuDNN Packages
|
|
|
|
If manual installation fails, try these alternatives:
|
|
|
|
### Option 1: Install via Conda (Recommended)
|
|
```cmd
|
|
conda install -c conda-forge cudnn
|
|
```
|
|
|
|
### Option 2: Use Docker with GPU Support
|
|
```dockerfile
|
|
FROM nvidia/cuda:12.0-runtime-ubuntu20.04
|
|
# Your LightRAG setup here
|
|
```
|
|
|
|
## Current Working Configuration (CPU Mode)
|
|
|
|
The system is fully operational in CPU mode:
|
|
|
|
```python
|
|
# Current working OCR configuration
|
|
from paddleocr import PaddleOCR
|
|
ocr = PaddleOCR(use_textline_orientation=True, lang='en') # CPU mode
|
|
```
|
|
|
|
## Performance Impact
|
|
|
|
- **CPU Mode**: ~2-3x slower than GPU
|
|
- **GPU Mode**: Full RTX 4070 SUPER acceleration
|
|
- **Current Status**: System works perfectly in CPU mode
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues:
|
|
1. **cuDNN not found**: Ensure files are copied to correct CUDA directory
|
|
2. **PATH not set**: Verify CUDA bin directory is in system PATH
|
|
3. **Version mismatch**: Ensure cuDNN version matches CUDA 12.x
|
|
|
|
### Verification Commands:
|
|
```cmd
|
|
# Check CUDA installation
|
|
nvcc --version
|
|
|
|
# Check GPU status
|
|
nvidia-smi
|
|
|
|
# Test PaddlePaddle GPU
|
|
python -c "import paddle; print(paddle.is_compiled_with_cuda())"
|
|
```
|
|
|
|
## Final Notes
|
|
|
|
- The LightRAG OCR system is **fully functional** in CPU mode
|
|
- GPU acceleration provides **performance improvement** but is not required
|
|
- Manual cuDNN installation is needed for GPU acceleration
|
|
- System will automatically use GPU once cuDNN is installed
|
|
|
|
For immediate use, the current CPU mode configuration provides complete OCR functionality for PDF processing. |