Submissions Failing solution, It's not your Policy! It's Docker!

Steps I did for success.

First remove old junk:

for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove -y $pkg done

#2 Check what’s left

docker --version
docker compose version

dpkg -l | grep -i docker

systemctl status docker

docker ps -a
docker images
docker volume ls
docker network ls

docker system df

rc = removed, config files remain only.

rc ``docker.io

old Python compose wrappers installed, but not real Docker Engine.

sudo apt purge -y ``docker.io`` python3-compose python3-docker python3-dockerpty

sudo apt autoremove -y

Then check again

dpkg -l | grep -i docker

Start Fresh!

sudo apt update
sudo apt install -y ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | 
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

sudo chmod a+r /etc/apt/keyrings/docker.gpg

Next add the Docker apt repo:


echo \
  "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update

Next install Docker Engine:

sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

docker --version
docker compose version

Next: non-root Docker access.

sudo groupadd docker

sudo usermod -aG docker $USER

Then apply it without reboot:

newgrp docker

Start Docker:

sudo service docker start

sudo service docker status

docker run hello-world

I used WSL so follow if you are.

Next tiny cleanup/test:

docker ps

Then check non-root access is really good:

groups

You should see docker in the group list.

if you have NVIDIA follow along:

First check NVIDIA is visible inside WSL:

nvidia-smi

if not fix it!

First verify package exists:

apt list --installed | grep nvidia-container

If nothing important shows up, run:

sudo apt install -y nvidia-container-toolkit

Then configure Docker runtime:

sudo nvidia-ctk runtime configure --runtime=docker

Then restart Docker:

sudo service docker restart

Now test GPU inside Docker:

docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi

If that prints the GPU table, NVIDIA Docker is good.

Thanks! Curious how did you figure out it is due to docker issue? is there any command to run to verify that the docker set up is incorrect?

Unfortunetly, after following the steps and doing a new submission it still failed. Everything seemed to work just fine until the submission portal returned the failed status with a blank stderr file. I would recommend though to also check if the Dockerfile and docker-compose.yaml (from ./docker/node-name) are up to date with the latest commit, or to do a git pull.

I uploaded a blank policy and it still failed with my old setup.