RuntimeError: cublas runtime error during training #64
Description
Hello,
I have encountered an issue while running the training script with the following error:
RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/THC/THCBlas.cu:249
The error seems to occur during the forward pass of the network:
File "/workspace/instagan/models/insta_gan_model.py", line 145, in forward self.fake_B_sng = self.netG_A(self.real_A_sng)
Here is my system configuration:
Operating System: Ubuntu 20.04.5 LTS
PyTorch version: torch==0.4.0. torchvision==0.2.1
CUDA version: release 11.8
I have tried various solutions, but the issue persists. Any help or suggestions on how to resolve this issue would be much appreciated. Thank you in advance!
Best regards,
Mario