ScaDS.ai-Logo

Live-share control files

Why control files?

Normally live-sharing is enabled when a run starts, with --live_share . Sometimes you only notice later that a run should be shared, or you want to stop sharing a job that is already running. Restarting the job just to change that is usually not an option.
OmniOpt2 therefore watches two marker files while it runs:
| File | Effect | | --- | --- | | liveshare | Enable live-sharing for the matching runs | | stop_liveshare | Disable live-sharing for the matching runs |
The files only need to exist. Their content is ignored.
touch liveshare

No restart is required. OmniOpt2 checks the marker files about once per second.

Where to put the files

The location of the marker file decides which runs it affects. More specific locations win when several scopes exist, but stop_liveshare always wins over liveshare .
| Location | Affects | | --- | --- | | runs///liveshare | Only this single run | | runs//liveshare | All runs of this experiment, including future ones | | liveshare in the current working directory | All runs, including future ones | | liveshare next to the omniopt script | All runs, including future ones |
For example, if the job is runs/my_experiment/0 :
# Only runs/my_experiment/0
touch runs/my_experiment/0/liveshare

# Every run of my_experiment (running and future) touch runs/my_experiment/liveshare
# Every run of every experiment (running and future) touch liveshare

The same paths work for stop_liveshare :
touch runs/my_experiment/0/stop_liveshare
touch runs/my_experiment/stop_liveshare
touch stop_liveshare

Deleting a marker does not undo it

The marker files are triggers, not permanent settings:
This is intentional: create the opposite marker when you want to change the state.
touch liveshare          # start sharing
rm liveshare             # still shared
touch stop_liveshare     # stop sharing
rm stop_liveshare        # still stopped
touch liveshare          # start sharing again

Choosing between scopes

Because stop_liveshare always wins, a global stop file pauses every run until it is removed, even if a more specific liveshare file exists. This makes the stop file a reliable emergency brake.
If you want to keep a global liveshare but stop only one experiment or run, use the more specific stop_liveshare file. The global share stays enabled for all other runs.

What happens in the terminal?

When a marker enables sharing, OmniOpt2 immediately uploads the current state and prints the share URL. While a tqdm progress bar is visible, the message is written above the bar, and the bar continues below it:
text
 60%|██████░░░░| 3/5 [00:00<00:00, 29.30it/s]
LiveShare activated (global marker: /home/me/OmniOpt/liveshare)
LiveShare URL: https://imageseg.scads.de/omniax/share?user_id=me&experiment_name=my_experiment&run_nr=0
 60%|██████░░░░| 3/5 [00:00<00:00, 29.30it/s]

The activation message is printed only once per transition, not on every poll. After the initial upload, live-share updates are pushed periodically, exactly like a run started with --live_share .

Relationship to --live_share

Caveats

The running OmniOpt2 process must be able to see the marker files. On a cluster, create them on the same shared filesystem that contains the OmniOpt2 installation and the runs folder. The current working directory of the process and the directory of the omniopt script are both checked.

Marker files are listed in .gitignore, so accidentally committing them is not a problem. They are meant to be temporary control files.

Sharing still requires network access to the OmniOpt2-Share server. If the server is not reachable, the activation message is printed without a URL and the periodic updater keeps retrying.

See also