# Upcoming maintenance for build.ros.org: 2020-12-23

**URL:** <https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107>\
**Category:** Buildfarm\
**Created:** [December 19, 2020, 1:10am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107 "2020-12-19T01:10:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![nuclearsandwich](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/nuclearsandwich/32/559_2.png) [@nuclearsandwich](https://discourse.openrobotics.org/u/nuclearsandwich)\
**Post date:** [December 19, 2020, 1:10am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/1 "2020-12-19T01:10:19Z")

</div>

[build.ros2.org](http://build.ros2.org) has been migrated to the new cookbooks and now it is [build.ros.org](http://build.ros.org)’s turn.

Learning from the last migration there may be multiple days of downtime if data transfer throughput is the same as last time although I’m going to try and ensure that our new instances are brought up in the same availability zone as the current ones to reduce the transfer overhead.

During the downtime [build.ros.org](http://build.ros.org) will be offline and pull request jobs will not be triggered. We’ll hold all ros/rosdistro release pull requests during the maintenance period so new releases won’t be merged during that time. I’ll send updates to [status.ros.org](http://status.ros.org) during the maintenance.

---

<div class="post-metadata">

**Author:** ![Aaron\_Blasdel](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/aaron_blasdel/32/4214_2.png) [@Aaron\_Blasdel](https://discourse.openrobotics.org/u/Aaron_Blasdel)\
**Post date:** [December 24, 2020, 1:57am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/2 "2020-12-24T01:57:14Z")

</div>

This would cause [repositories.ros.org](http://repositories.ros.org) to be inaccessible correct? Any rough ETA for completion and repositories to be back online?

---

<div class="post-metadata">

**Author:** ![nuclearsandwich](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/nuclearsandwich/32/559_2.png) [@nuclearsandwich](https://discourse.openrobotics.org/u/nuclearsandwich)\
**Post date:** [December 24, 2020, 2:53am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/3 "2020-12-24T02:53:55Z")

</div>

[repositories.ros.org](http://repositories.ros.org) is offline but [packages.ros.org](http://packages.ros.org) is unaffected and is where most consumers should be retrieving packages.

The repositories host will probably be back online some time tomorrow once data transfer from the old host is complete.

---

<div class="post-metadata">

**Author:** ![nuclearsandwich](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/nuclearsandwich/32/559_2.png) [@nuclearsandwich](https://discourse.openrobotics.org/u/nuclearsandwich)\
**Post date:** [December 24, 2020, 4:27pm UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/4 "2020-12-24T16:27:10Z")

</div>

The repositories host has been migrated to a new instance. There are some cosmetic differences between the auto-index of nginx and apache but the repository contents should match the last state of the previous repository host before it was taken offline.

---

<div class="post-metadata">

**Author:** ![mikeferguson](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/mikeferguson/32/211_2.png) [@mikeferguson](https://discourse.openrobotics.org/u/mikeferguson)\
**Post date:** [December 29, 2020, 9:17pm UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/5 "2020-12-29T21:17:30Z")

</div>

So I’m not sure if this is related - but I’m also not sure where else to post an issue with the build farm. It seems since this migration, our PRs are failing to build for the navigation stack - because they can no longer download resources from [download.ros.org](http://download.ros.org).

example of a recent failure: [https://build.ros.org/job/Npr\_\_navigation\_\_ubuntu\_focal\_amd64/53/console](https://build.ros.org/job/Npr __navigation__ ubuntu_focal_amd64/53/console)

Any ideas of what might be the cause, or how to solve this?

-Fergs

---

<div class="post-metadata">

**Author:** ![nuclearsandwich](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/nuclearsandwich/32/559_2.png) [@nuclearsandwich](https://discourse.openrobotics.org/u/nuclearsandwich)\
**Post date:** [December 29, 2020, 10:08pm UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/6 "2020-12-29T22:08:49Z")

</div>

> [@mikeferguson](#):
>
> they can no longer download resources from [download.ros.org](http://download.ros.org).

Thanks for the report. I actually just got the same report internally from @sloretz and @jacob

When setting up the new cluster used a private DNS zone within AWS to route traffic on internal interfaces and that zone is interfering with public DNS for other [ros.org](http://ros.org) domains. To mitigate the issue for now I’ve been adding matching records for required domains to the private zone but we’re going to look at switching to subdomain-specific private zones which take a bit more configuration duplication but remove the need to mirror the public DNS for other subdomains.

I’ve updated the private zone to include [download.ros.org](http://download.ros.org) so subsequent builds should not have this issue.

---

<div class="post-metadata">

**Author:** ![mikeferguson](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/mikeferguson/32/211_2.png) [@mikeferguson](https://discourse.openrobotics.org/u/mikeferguson)\
**Post date:** [December 30, 2020, 1:57am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/7 "2020-12-30T01:57:05Z")

</div>

@nuclearsandwich thanks for the quick response - can confirm that our jobs are now passing again!

---

<div class="post-metadata">

**Author:** ![nuclearsandwich](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/nuclearsandwich/32/559_2.png) [@nuclearsandwich](https://discourse.openrobotics.org/u/nuclearsandwich)\
**Post date:** [December 31, 2020, 12:22am UTC](https://discourse.openrobotics.org/t/upcoming-maintenance-for-build-ros-org-2020-12-23/18107/8 "2020-12-31T00:22:04Z")

</div>

I’ve marked this maintenance window as over! Thanks everyone for the patience and feedback. Do continue to report issues in this thread or the [ROS 2 one](http://discourse.ros.org/t/upcoming-maintenance-for-build-ros2-org-2020-11-25/17003/11) if you find one. I’m about to make a separate post about the pull request web hooks.