Out of tree build with goa

Developer's Corner
1

Hi,

I am running goa from a higher-level build harness that drives many build flavours of the same component (different boards, different kernels, different patch series applied to the cached Genode source). Each flavour resolves to its own out-of-tree cache directory, and the harness invokes goa build + goa export once per flavour.

What I’d like is a way to tell goa, on the command line, where to put var/ for each invocation, so two flavours of the same project can build side-by-side without stepping on each other.

What I tried so far:

  • set depot_dir <path> and set build_dir <path> in a project-localgoarc. Works fine for a single flavour, but the path is baked intothe file — the harness can’t vary it per invocation without rewriting the goarc each time (and any concurrent invocation racing on that rewrite is a recipe for fun).
  • set common_var_dir <path> in ~/goarc. Closer to what I want:
    goa places var/ under <common_var_dir>/<project-name>/. But the key is the project name only — two builds of the same component at the same time still collide.
  • Hierarchical goarc lookup (walking up from the project dir). That works for setups where each flavour lives under a distinct parent directory, but our flavours all consume the same source tree from different cache trees, so there is no natural per-flavour parent to
    hang a goarc off of.
  • didn’t try to sacrifice a goat yet (sacrifical goats have been in short supply recently :wink: )

What I observe when two flavours collide on the same var/ (e.g. both running goa build against the same src/<project>/<component>/ while their cache trees differ): goa’s pre-build var/depot rewrite trips on a half-populated subdir from the sibling invocation, fails
with OSError: [Errno 39] Directory not empty: <archive> and unwinds. I can serialise the harness as a workaround, but on a 13-component matrix that costs a lot of wall time per build round.

Questions:

  • Is there an existing way to point var/ (or equivalentlybuild_dir + depot_dir) to an absolute path via the command line? Something like goa build --common-var-dir <abs-path> or goa build --build-dir <abs-path> --depot-dir <abs-path>.
  • If not, would a patch adding one be welcome? I’d be happy to contribute it. The shape I have in mind is a simple CLI override that, when present, supersedes whatever goarc resolved.
  • Alternatively, would honouring an environment variable (e.g. GOA_COMMON_VAR_DIR) be more in keeping with goa’s style?

Thanks!

PS:

happy to share the reproducer if useful — it’s just two goa buildinvocations on the same component with different cache trees and no per-flavour goarc separation.

2

Yes, that is already possible exactly as you proposed. I must admit that the documentation currently fails to mention this.

3

Thanks @jschlatow!

Confirmed by reading the source — for anyone else who lands here:

TL;DR: yes, goa build --common-var-dir <abs-path> (or --build-dir / --depot-dir, or any of the other *_dir config variables) works today, the docs just don’t mention
it.

The mechanism is in share/goa/lib/config.tcl and share/goa/lib/command_line.tcl. Every config variable whose name ends in _dir is auto-registered as a CLI switch, with
underscores rewritten to dashes:

# share/goa/lib/config.tcl
namespace eval config {
    ...
    variable common_var_dir           ""
    variable depot_dir                ""
    variable build_dir                ""
    variable public_dir               ""
    variable run_dir                  ""
    ...

    # return names of path variables
    proc path_var_names {} {
        return [info vars ::config::*_dir] }
    ...
}

# share/goa/lib/command_line.tcl
foreach var_name [config path_var_names] {

    set var_name [lindex [split $var_name :] end]
    regsub -all {_} $var_name "-" tag_name

    set path [consume_optional_cmdline_arg "--$tag_name" ""]

    if {$path != ""} {
        set config::$var_name [file normalize $path]
        lappend allowed_paths [set config::$var_name]
    }
}

So the available switches are (one per variable):
--common-var-dir, --search-dir, --depot-dir, --public-dir,
--contrib-dir, --contrib-cache-dir, --import-dir,
--abi-dir, --build-dir, --run-dir, --hsd-dir,
--bin-dir, --dbg-dir, --install-dir.

And when set, they bypass the per-project defaults in config::set_late_defaults:

set var_dir [file join $project_dir var]
if {$common_var_dir != ""} {
    set var_dir [file join $common_var_dir $project_name] }

proc set_if_undefined { var_name value } {
    upvar ::config::$var_name var
    if {![info exists var] || $var == ""} {
        set var $value }
}

set_if_undefined depot_dir         [file join $var_dir depot]
set_if_undefined build_dir         [file join $var_dir build $arch]
...

I’ll wire --common-var-dir <per-flavour-abs-path> into my build harness and report back if I hit anything unexpected. If it works end-to-end, I’d be happy to send a small docs PR for
goa help config so the next person finds these without having to grep the source.

=> Back to Hacking mode :slight_smile:

1 Like
4

That would be great!

5

Quick follow-up, I was able to make it work as described.

The only caveat I hit: <common_var_dir>/<project_name> is goa’s var_dir, not its parent — meaning depot/, build/, bin/ etc. live directly under <project_name>/, not under a nested var/. Once I matched my mirror path to that layout it was a clean switch.

I see you just merged the PR already: https://codeberg.org/genodelabs/goa/pulls/177

Thanks!