Dear AIC Team,
After building the docker container like the Submission.md docs mentioned and testing it locally, it spun up the containers and the evaluation worked well. When I then uploaded and submitted it on the website, it failed without explanation and logs. Is there a chance to perhaps receive some more information regarding this, or more detailed logs to debug why it may have not worked on the the submission portal.
Thank you!
+1 on this. The current result and stdout files do not have anything in them.
Hi, could you provide us your team name?
my team is learning2boost
my team name is Forgis
Hello we have the exact same problem. Our team is called:
mission-assembly-r0b-protocol
same issue, I used podman instead of docker. It worked well while running locally. Team tcs-plgrm
same issue Team name: TCS-PLGRM
I saw you manage to send the submission Mission Assembly - R0B Protocol. What did you change?
We’ve just rolled out changes that make the eval logs available for your submissions. The logs could provide some insights on why the submissions are failing. For example, a common case is that the AIC engine was timing out when waiting for the submitted model to be ready (see timeout limits in aic/docs/challenge_rules.md at main · intrinsic-dev/aic · GitHub )
You should see be able view these logs in the stdout files in your next submission.
i just submitted and it failed, and the logs and the err files are blank, i had tested the image locally and it worked
We just uploaded the wavearm policy to see if that would work. We haven’t tried uploading our own again.
Following the recent rollout, we have observed a regression in the evaluation script. Specifically, the system is failing to wait for the aic_model node to transition from the ‘unconfigured’ state to ‘active,’ resulting in an immediate timeout error.
While this issue was resolved in a prior version, it appears the fix may not have been captured in the current eval image on the submission portal. This likely accounts for the widespread failures reported by other users. Could we verify that the latest stable changes are correctly reflected in the portal’s environment?
I also tried again, same, no logs
have u tried pushing again?
Every transition for a node has 60 second timeout. If your node takes more than 60 seconds to configure, where different model initialisation happens, it will trigger timeout error.
I removed the warm-ups which i was doing earlier so that moving from configure to activate takes less than 60 seconds.
Currently, my submission is no longer failing; however, I am receiving a score only for Trial 1. The stdout file does not contain any logs from the aic_model node, which is making it difficult to identify the root cause of the issue.
Due to the absence of these logs, I am unable to determine the possible error or understand why the subsequent trials are not being evaluated as expected. Our team name is TCS-PLGRM.
Could you please look into this and let us know
I am dealing with the same issue and would appreciate any visibility and insights. I have had 5 failed submissions: Runs 207, 236, 238, 255, 256 , 5 different images, 5 different timing profiles (637 → 296 → 185 → 132 → 126 s), all with zero log output. All these images work perfectly fine locally and even on another machine for a sanity check. Team name is DevicetoCloud.
Is it possible to get access to container logs? I have also filed a issue on github if thats helpful.
Thank you.
I have written a post explaining the problem and the fix that worked for me (not using sim time and deferring imports to init) have u tried those things?
Do we need to explicitly end the task before the timeout mentioned in the task board.
