menu

Mutagen

Mutagen makes remote development work with your local tools

Channels
Team

Orchestrating and embedding Mutagen

July 15, 2019 at 4:43am

July 19, 2019 at 9:39am
Hi , have a question about how mutagen behaves in case of multiple instances.
For example, we have mutagen executable not system-wide, but in our project's scope. We've using it to create sync sessions exclusively with our docker containers.
How the sync sessions will behave if the system has system-wide mutagen already installed (via homebrew for example). If there was also active other sync sessions started with that system mutagen?
Basically, we are thinking about scoped mutagen instance because we will need to control versioning and we don't want to touch system configuration (for cases if it is used in other projects).
Thanks in advance!
like-fill
1
  • reply
  • like

July 20, 2019 at 6:28am
Hey , at the moment there's no "scoping" based on executable - everything is attached to the single user-global Mutagen daemon. The closest thing to scoping in v0.9.x is using labels to scope/select sessions, but I think what you're asking for is the ability to maintain a separate Mutagen daemon on a per-project basis, correct?
What if I were to add an envionment variable like MUTAGEN_DATA_DIRECTORY that tells Mutagen to look somewhere other than ~/.mutagen for the daemon socket and data? You could then do:
MUTAGEN_DATA_DIRECTORY=/some/directory mutagen daemon start
Or, if you just want to host the process yourself:
MUTAGEN_DATA_DIRECTORY=/some/directory mutagen daemon run
Then when you script commands, it would look like:
MUTAGEN_DATA_DIRECTORY=/some/directory mutagen sync create ...
Would that work? I'd prefer to do this as an environment variable since (a) it would be a little experimental and (b) it avoids needing to add --data-directory flags to every command.
The agent binaries and data on remote systems would still go into ~/.mutagen on those systems, but that's fine since that data is designed to co-exist with multiple clients.
like-fill
3
  • reply
  • like

July 23, 2019 at 2:29pm
Thanks for the informative reply !
ability to maintain a separate Mutagen daemon on a per-project basis, correct?
Yep. In our case, our toolset will manage many projects which will be controlled by sync session labels. The idea was to have a separate mutagen executable in the scope of our toolset.
A configurable environment variable is a good option, all commands will be scripted anyway.
Will it allow us to start/stop mutagen daemon separately from another, possibly, active mutagen instance? And control only sync sessions which were created by "our" mutagen instance. So for example command mutagen daemon stop won't break "system" mutagen sessions and daemon.
btw mutagen daemon run command is not documented yet, I never heard about it before. :)
Edited
  • reply
  • like

July 25, 2019 at 9:25am
Hey , I've just tagged v0.10.0-beta2, which allows the type of embedding that you want to do. The documentation for v0.10.x is going out tomorrow on the new website (stay tuned), but there's a quick summary of the changes in the release notes for v0.10.0-beta1.
Anyway, what you can now do is set the MUTAGEN_DATA_DIRECTORY environment variable to an absolute path that will serve as the data directory and IPC endpoint for an isolated daemon instance. It won't have any effect on other Mutagen daemon instances. When it comes to embedding, the best way to do it is with the aforementioned mutagen daemon run command, which is a hidden command that serves as the entrypoint for the daemon (i.e. it doesn't background). So your toolset would do something like:
MUTAGEN_DATA_DIRECTORY=/some/internal/directory mutagen daemon run
Of course, you can still use the normal daemon start and stop commands to create a backgrounded daemon instance, but if you're embedding the daemon, this is probably easier to control. To communicate with that daemon, you just specify the same environment variable value to other Mutagen commands, e.g.
MUTAGEN_DATA_DIRECTORY=/some/internal/directory mutagen sync create ...
or
MUTAGEN_DATA_DIRECTORY=/some/internal/directory mutagen daemon stop
One thing worth noting about the v0.10.x release is that the commands have been restructured a little bit. Since Mutagen's functionality is expanding, the synchronization session commands now live under mutagen sync. However, to keep scripts from breaking, I've left the original create/list/flush/pause/resume/terminate available as hidden aliases under the root command, so mutagen create and the other commands still work.
If you get a chance to play with v0.10.0-beta2, I'd really appreciate any feedback. There's a lot of new stuff. I wouldn't ship it out or announce it to your users yet, but hopefully it won't be long until the final release is tagged. I'd like to make that happen this week.
Edited
like-fill
2
  • reply
  • like

August 20, 2019 at 3:51pm
Hello !
I'm integrating mutagen v0.10.0 to our project and everything great and works smoothly so far.
We are using the new orchestration feature and faced with small limitation, it's not possible to run flush command for the exact session with mutagen project flush command. Maybe there is another way to flush separate session in project scope? Or it's better to name sessions with project prefix and use mutagen sync flush <session_name>?
Thanks!
  • reply
  • like

August 20, 2019 at 9:58pm
Hey ! The synchronization (and forwarding) sessions created by the orchestration infrastructure are still normal sessions, they just have an additional io.mutagen.project label attached to them. This means that they can still be flushed using the mutagen sync flush command by using their session ID (or by name, though names may not be unique). But I'm thinking now that maybe it would be helpful to add label support to the project configuration? I.e. if you could assign labels for each session in the mutagen.yml file, you could more precisely specify them using the mutagen sync flush command. Would that be helpful?
  • reply
  • like
Basically, labels are already assigned in the project's mutagen.yml, like
sync:
labelName1:
alpha: ...
beta: ...
...
labelName2:
...
In our case, if we reuse mutagen.yml for different projects, mutagen sync flush labelName1 will affect all active projects sync sessions with name labelName1? I think command like mutagen project flush labelName1 would be good to flush only labelName1 session of the specific project.
  • reply
  • like
So the keys below sync: are actually session "names" rather than labels, but you're correct that they aren't unique and that using mutagen sync flush <name> may very well affect sessions outside of the project. I was proposing adding a labels key to the session configuration, but actually your solution is better. Internally, the project orchestration can map something like mutagen project flush <name> to a <name> && io.mutagen.project=<project-id> requirement. This will take a bit of work, but it is more elegant. I'll try to draft up a proposal for that and get back to you.
like-fill
1
  • reply
  • like
Yes, you're absolutely right. I was talking about session names, don't know why I've named them as labels, as we have labels with project identifiers.
I'll try to draft up a proposal for that and get back to you.
Many thanks! This orchestration thing becomes very handy for reusing and scaling.
  • reply
  • like