git Gentoo repos on GitHub

git me! The gentoo repos are on github, and I've just discovered them.. mysteriously hidden in the World Wide Web ;)

There is a post on the gentoo forums from 2015, re git/github saying:
Although some developers wrongly still recommend plain rsync-update method, you should never do that, since it is fundamentally insecure (emerge-webrsync is signed by a machine key, but still better). 

You should instead start to sync via git (best do that from github, unless you want to break our infra server).

Essentially, rsync update method is outdated and insecure. The recommendation is to sync update via git, and more specifically github, rather than the gentoo infra servers.

GitHub gentoo (without metadata cache, for developers) repo:

This seems to be a mirror of the gentoo git infra server:

I found a github mirror, which I now use. For anonymous portage sync updates, I am hoping this secondary mirror may be more stable and less noisy.

GitHub gentoo-mirror (with metadata cache, for users) repo:

Before we change our sync method to git, we need to prepare/clean the portage location. Delete existing files, or move them elsewhere, in case you might have second thoughts ;) If you are not happy with git, you can move them back, and continue with rsync updates.

# mv /usr/portage /tmp

If you don't have enough freespace in /tmp, or don't want to use it, move it elsewhere and remember the location.

My /tmp is on tmpfs, and I have enough free space there for now. If you have a similar setup, note that tmpfs will not survive a reboot.

Now we can change our sync method to git.

# vi /etc/portage/repos.conf/gentoo.conf
main-repo = gentoo

location = /usr/portage
sync-type = git
sync-uri = git://
auto-sync = yes

Make sure there are no other files in that folder. I had trouble when I renamed my gentoo.conf file to another name in that same folder as backup. I changed auto-sync = no to be safer. Sync wouldn't give me any errors, or updates!

I previously used the gentoo infra server, before changing over to github, and then to the mirror. Apparently, the gentoo infra servers are not ready to take the weight of the whole world. Apparently, GitHub is safer and more reliable. I was pleasantly surprised that my git was in sync after the changeover.

We have prepared, cleaned and emptied our portage for a git sync. Now we can sync.
# emerge --sync

And move the distfiles back where they belonged.
# mv /tmp/portage/distfiles /usr/portage/

Now you can sync again and again.. to your heart's content :) No more worrying about overloaded rsync servers warning, or ending up on the ban list for syncing too much.

It is now a pleasure to sync, and see exactly what has changed where. Such a difference from watching that cruft fly past like a matrix on your screen. You might miss the filed Change Logs, if you used them. They are no longer needed. Git has much better, automated and less error prone methods.

Note the efficiency of git sync, and github :) This is what Linus alluded to, when he wanted to have an efficient kernel repository, accessible to the whole world.

If you are happy with git sync, you can delete that portage backup.
# rm -rf /tmp/portage

Another more important aspect is about making open source truly open! Anyone, even without github logins, can view the sources and changes. With a free github account, anyone could make suggestions via a simple pull request. You don't even need to be a developer.

No comments:

Post a Comment

most popular posts