Lookaside/Git

Lookaside/git¶

This page documents how SIGs can upload the sources of their RPM into the lookaside cache and how they should set-up their corresponding git repository.

It should be noted that two model exists, the "traditional" model and the one based on CentOS Stream and Fedora. While both model can be intertwined in some places, it is recommended that SIGs pick a model and follow it. This documentation will not cover which aspect of which model is compatible with the other one.

New Package (from source)¶

If your SIG uses https://gitlab.com/CentOS, you, your SIG chair or people that have been granted access by the SIG chair, will be able to create the project for your package directly.

In the rest of the document we will use the following example : Build the package centpkg-minimal, as a member of the sig-core SIG group. We have to push two things:

the .spec file to build the rpm, to the git repository
eventually an archive (.tar.gz, .tar, .xz...) of the sources, to lookaside cache

Warning

To push to lookaside cache you need to have already a local check out from centos-git-common , or better, install the centos-packager rpm package, which provides centos-lookaside-upload and centos-lookaside-upload-sig scripts

Pushing first to lookaside cache¶

Let's assume that my pkg centpkg-minimal that I want to build has an archive called centpkg.minimal.tar.gz. To be able to push to lookaside cache, we need the following :

a valid TLS cert (also needed to build on cbs.centos.org) obtained through centos-cert util
either script lookaside_upload or lookaside_upload_sig in your $PATH (coming from centos-git-common git repo, see above), or centos-lookaside-upload / centos-lookaside-upload-sig (from centos-packager rpm, available on Fedora and Stream through epel)

There are two path available for the lookaside cache, each is available using a different script:

lookaside_upload allows uploading to the traditional CentOS Linux structure: baseurl/pkgname/branch/hash (example)
lookaside_upload_sig allows uploading to the same lookaside structure as the one used by CentOS Stream and Fedora: baseurl/pkgname/tarball/hashtype/hash/tarball (example)

Both scripts need some paramters, simply use -h to see them.

Back to our example.

Pushing to the "traditional" lookaside structure¶

The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz and I'm member of the sig-core group, and want to build it for c7, so we'll call it like this :

lookaside_upload -f centpkg-minimal.tar.gz -n centpkg-minimal -b c7-sig-core
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
[+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
[+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
[+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
[+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======

################################################################################################################ 100.0%
File centpkg-minimal.tar.gz size 15178 CHECKSUM d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 stored OK
[+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
[+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/centpkg-minimal/c7-sig-core/d6616b89617914a0dd0fd5cfa06b0afc7a4541c4

Pushing to the CentOS Stream/Fedora lookaside structure¶

The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz, so we'll call it like this :

lookaside_upload_sig -f centpkg-minimal.tar.gz -n centpkg-minimal
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
[+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
[+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
[+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
[+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======

################################################################################################################ 100.0%
File centpkg-minimal.tar.gz size 15178 CHECKSUM d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 stored OK
[+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
[+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/centpkg-minimal/c7-sig-core/d6616b89617914a0dd0fd5cfa06b0afc7a4541c4

Now that we have uploaded to lookaside cache, we can reference it in our /rpms/centpkg-minimal git repository on git, see below

Pushing to git¶

gitlab.com/CentOS

If your SIG wants to use gitlab, they will have to follow the instructions to request one. Here we'll cover both workflow, assuming the git repository exists on either platform.

Git clone¶

The first thing you'll need is a local checkout of the git repository for the pkg we'd like to work on.

So the way to git clone/pull over ssh is git@gitlab.com:CentOS/<sign>/rpms/<repo>.git

Git branches¶

Since SIGs have full control over their namespace on gitlab.com, that branching structure is not mandatory. It will, however, be necessary if you plan on using the "traditional" lookaside structure describe above as this relies on branch names.

Git repository layout¶

There are two possible structure possible for the repository layout:

The exploded SRPM layout
The flat dist-git layout

The exploded SRPM layout¶

This is the "traditional" layout used in CentOS Linux, it is articulated around two folders:

├── SOURCES
└── SPECS

The SPECS folder is meant to receive your spec file and all other text files and patches would go under SOURCES:

├── SOURCES
│   ├── <optional_file>
│   └── <optional_patch>
└── SPECS
    └── <pkg_name>.spec

Once done, we still need to link these files with the sources uploaded earlier in the lookaside cache. See the next section.

The flat dist-git layout¶

This is the layout used in the dist-git repositories in CentOS Stream and Fedora. All files (spec files, text files, patches...) are stored at the top level of the repository:

├── <optional_file>
├── <optional_patch>
└── <pkg_name>.spec

Linking to sources in the lookaside cache¶

Here as well, there are two ways you can link a dist-git repository to sources uploaded to the lookaside cache:

the "traditional" way using a .<pkg_name>.metadata file
the CentOS Stream/Fedora way using a sources file

Important

Even if your package doesn't contain any source pushed to lookaside cache (like for a package just having some small files in SOURCES/ dir, you need to have either a .<pkg_name>.metadata or sources file present and pushed in git repository

Using a `.<pkg_name>.metadata` file¶

To use this method, simply create a .<pkg_name>.metadata file with the following content:

<sha1 of the tarball> SOURCES/<filename>

For example:

d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 SOURCES/centpkg-minimal.tar.gz

Using a `sources` file¶

To use this method, simply create a sources file. The easiest way to achieve this is:

sha512sum --tag <tarball> > sources

Example output:

SHA512 (centpkg-minimal-1.1.0.tar.gz) = f7cfbc956199b1a0342f321a0e4cb055d6ac2c784a8faace43cf80772f0de34b58ede1a02a558cd03e25c14938ac6e17b3746b1919c0bbc1f5b2955472577e4f

Now that we have pointer to lookaside cache, and also .spec, we can push back to git and we should be able to proceed with the "build-from-git" on cbs.centos.org. Let's commit first.

Git push¶

Depending on if you picked the "traditional" structure or the CentOS Stream/Fedora one, this is how your repository should look like:

Traditional layout

.
├── .<pkg_name>.metadata
├── SOURCES
│   ├── <optional_file>
│   └── <optional_patch>
└── SPECS
    └── <pkg_name>.spec

CentOS Stream/Fedora layout

.
├── sources
├── <optional_file>
├── <optional_patch>
└── <pkg_name>.spec

We can now push to to git, as usual:

# git add <files> # if needed
git commit -a
git push origin <branch> # to create the remote branch if not existing yet

Where <branch> would be c7-sig-core for our example using git.centos.org.

Now that we have our sources pushed to both src.sigs.centos.org and lookaside cache, we can now proceed with a build in cbs/koji.

Lookaside/Git

Lookaside/git¶

New Package (from source)¶

Pushing first to lookaside cache¶

Pushing to the "traditional" lookaside structure¶

Pushing to the CentOS Stream/Fedora lookaside structure¶

Pushing to git¶

Git clone¶

Git branches¶

Git repository layout¶

The exploded SRPM layout¶

The flat dist-git layout¶

Linking to sources in the lookaside cache¶

Using a .<pkg_name>.metadata file¶

Using a sources file¶

Git push¶

Using a `.<pkg_name>.metadata` file¶

Using a `sources` file¶