Subject: suggesting feature, splitting pkgs repositories (part II)

From: "V.Krishn" <[email-masked]>
Date: Thu, 19 Jan 2023 23:08:31 +0530

Hi!

Would like to suggest moving -dbg pkgs in repositories into a separate 
folder. eg.
main/aarch64.dbg or community/aarch64.dbg/
OR
main.dbg/aarch64 or community.dbg/aarch64

Suggested change is desirable because:
1. Users who would want to install -dbg pkgs are willing to do 
debugging. They are mostly development oriented.
2. Mirrors may opt to ignore these .dbg folder, thereby saving some disk 
space.

--
Regards,
V.Krishn

---

From: Konstantin Shelekhin <[email-masked]>
Date: Thu, 19 Jan 2023 19:47:19 +0300

> 2. Mirrors may opt to ignore these .dbg folder, thereby saving some disk

Mirrors can do that: rsync ... --exclude -*-dbg.apk' ...

---

From: Kevin Daudt <[email-masked]>
Date: Thu, 19 Jan 2023 17:49:16 +0000

> Mirrors can do that: rsync ... --exclude -*-dbg.apk' ...

The index will still list those packages, so users would be confused
where it looks like they can install the packages, but then not
available on the mirror.

---

From: Konstantin Shelekhin <[email-masked]>
Date: Thu, 19 Jan 2023 20:01:16 +0300

> where it looks like they can install the packages, but then not

Yeah, makes sense.

---

From: Kevin Daudt <[email-masked]>
Date: Thu, 19 Jan 2023 20:14:33 +0000

On Thu, Jan 19, 2023 at 11:08:31PM +0530, V.Krishn wrote:
> 1. Users who would want to install -dbg pkgs are willing to do debugging.
> They are mostly development oriented.

The question is if it is possible to move subpackages to a different
repository as the origin package.

---

From: "V.Krishn" <[email-masked]>
Date: Fri, 20 Jan 2023 14:26:13 +0530

> repository as the origin package.

Would need minor changes.
1. change REPODEST temporarily.
2. move -dbg pkgs to .dbg folder.
3. move rest to normal folder.
4. re-generate index in each folder.

But real hurdle could be more, and can be pointed out by devs running 
the build servers.

-- 
Regards,
V.Krishn

---

From: Timo Teras <[email-masked]>
Date: Sat, 21 Jan 2023 09:57:17 +0200

On Thu, 19 Jan 2023 20:14:33 +0000
Kevin Daudt <[email-masked]> wrote:

> > Would like to suggest moving -dbg pkgs in repositories into a
> > separate folder. eg.

This is not possible with changing apk code.

> > main.dbg/aarch64 or community.dbg/aarch64

This or some similar option like main-dbg, or dbg/main would be
possible without touch apk code.

> > debugging. They are mostly development oriented.

This would mean that new repositories need to be added if user wants to
use debug packages. Or perhaps adding some new apk feature to make
enablign the debug repos possible some other way.

> > 2. Mirrors may opt to ignore these .dbg folder, thereby saving some
> > disk space.

Understand. These can be fairly large at times. And it would be also
possible to configure close mirror without -dbg and another mirror with
the -dbg packages.

As an additional benefit the repository indexes without dbg packages
would be smaller - possibly helping embedded systems. And as mentioned
allow perhaps turning -dbg on by default (assuming we have enough disk
space on main mirrors).

> repository as the origin package.

From apk-tools side there is no problem. Having subpackages in a
separate repo is perfectly ok.

The complication comes from the building side. It would need special
logic to move the -dbg pacakges to different repository. And changes to
rebuild multiple indexes, and uploading to right place.

abuild also has assumptions that all generated packages are in same
destionation. This applies to detecting whether a rebuild is needed,
and on removing obsoleted packages.

All of the above should be doable, but might be non-trivial in some
cases.

I think the idea is good. But am not sure if/when someone would have
time to work on it. Would someone be willing to do the work if no one
objects for the change?

Timo

---

From: "V.Krishn" <[email-masked]>
Date: Sat, 21 Jan 2023 15:22:03 +0530

> This would mean that new repositories need to be added if user wants to

If user are installing -dbg pkgs, it can be expected to know things 
around apk, /etc/apk/.. and the ability to add a line or two in 
/etc/apk/repositories

> enablign the debug repos possible some other way.

Making it simple is always welcomed, but that would first need 
formalizing on variables like repo path .. etc. Maybe modifying apk 
could wait.

> would be smaller - possibly helping embedded systems.

Already happening.

  And as mentioned
> time to work on it. Would someone be willing to do the work if no one

Thanks for clarifications.
Looking forward in positive way :)

-- 
Regards,
V.Krishn

---

From: "alice" <[email-masked]>
Date: Fri, 20 Jan 2023 10:47:35 +0100

> debugging. They are mostly development oriented.

this is a downgrade because you now need to add more repositories to install
-dbg packages. this is, of course, assuming it's even possible to split
subpackages into another repository in the first place.

> 2. Mirrors may opt to ignore these .dbg folder, thereby saving some disk 

they can already do this with excludes passed to the rsync command.

so, what is the benefit?

1) isn't one (more steps to install -dbg packages for users, no upside?)

2) is already true

...2a) the reason not every package has -dbg is because of master mirror space
limitations- but how the folders are structured doesn't change the fact the
master mirror will have every -dbg package on it. the reason not everything has
a -dbg is not related to third party mirrors whatsoever. of course, being
able to host packages on more than one master server is a separate engineering
problem, and would be nice for reasons not related to this proposal (and would
also make it redundant anyway).

as far as i can see, this has *zero* practical upside, for any party involved:
the master mirror, third party mirrors, or users.

i guess you want this to have more -dbg packages (though you don't mention
that); getting those is not impacted by this extra repository layout at all.