Posted by @aaronchongth:
Thanks for confirming. In this case, the first few sanity checks would be to ensure RMW_IMPLEMENTATION and ROS_DOMAIN_ID on both pods are the same. Ideally using CycloneDDS, as we have experienced the least amount of issues with it
Posted by @Villy01:
As you can see in the previous logs, the ROS_DOMAIN_ID is the same in both pods, I also checked the RMW_IMPLEMENTATION and it is the same.
Posted by @aaronchongth:
I see, sorry I missed it at the bottom of the logs. I am unsure the cause of this at the moment. Perhaps some guides online can help with debugging communication between pods, https://serverfault.com/questions/1079368/pods-in-kubernetes-can-not-communicate-with-other-pods-and-outside-cluster-hosts
Posted by @Villy01:
The issue with pods not communicating with each other was due to some of them having the directive hostNetwork: true. By removing it, all the pods can now see and communicate with each other. I was therefore able to display the map on the dashboard.
I tried to load the map of my office into the system, and a question arose. To load it, I inserted it into the Docker image of the building map server pod and modified the map path so that it points to my file. My question is how to load the map without having to perform this operation. For example, if I make changes to my map file, I wouldn’t be able to see the modifications on the dashboard without redeploying the system, but restarting it would cause the changes to be lost. The only way to modify the map file is to load the modified file into the Docker image, rebuild it, and then redeploy. Have you thought of a more efficient way to manage this situation?
Posted by @akash-roboticist:
The only way to modify the map file is to load the modified file into the Docker image, rebuild it, and then redeploy.
No true - if you are using docker compose bind mounts allow you to map files into the container and similarly ConfigMap in the k8s have a similar functionality.
The template infact uses this feature in the chart to reference map files on L62-64
Posted by @Villy01:
The problem is that it seemed to me that updating the file in the Docker image or in the pod while it is running does not reflect any changes in the dashboard. Is there a sequence of actions to follow to update the file of a map while the pod is running and allow the changes to be displayed after refreshing the dashboard?
Posted by @aaronchongth:
Yeah, this is more in the scope of kubernetes, once the new docker image has been built/updated and is pulled to your local registry, you’ll generally just need to restart the pod
kubectl get pods -A
# get the name of the pod and namespace
kubectl delete pod -n <pod_namespace> <pod_name>
the cluster will spin up a new pod with the new image, assuming the image tag is the same.
If the image tag needs to be updated, after updating it, you’ll need to do a helm upgrade rmf rmf-deployment -n rmf (I think it’s the correct command, you’ll need to test), which updates the charts, then deleting any pods now should trigger a new pod starting with the new and correct image tags. You can further verify the running image tags with kubectl describe pod -n <pod_namespace> <pod_name>
Posted by @Villy01:
Thank you very much for the response, now it’s clearer to me. I have another question: is it possible to add a section in Grafana to view the RMF logs?
Posted by @aaronchongth:
Yeah it definitely is possible! It’s not on our roadmap yet, and we currently do not have the bandwidth to set this up, but contributions are always welcome! If there’s a grafana plugin that your team is interested in developing, and if there are any gaps for the implementation to happen, you can come by our PMC or community sessions to have a chat!