menu
announcement

Spectrum is now read-only. Learn more about the decision in our official announcement.

gigantum

Community for help, feedback, and all things Gigantum

Channels
Team

How to add new base environments?

March 2, 2021 at 9:28am

How to add new base environments?

March 2, 2021 at 9:28am
How do I add new base envs for projects to my gigantum client? I saw that there are docker images with different base envs, but I don't know how to install them. I am self hosting the gigantum client on a digital ocean droplet

March 2, 2021 at 2:24pm
If you can start from the base environments, you can edit the environment using apt, pip, and conda package managers from the environment tab of any project. You can also write custom docker instructions under the advanced settings section.
If you want to add your own base environments you can by creating a git repo somewhere (e.g. github) that stores metadata files about your bases (that you've manually built and pushed to a public registry (e.g. dockerhub). You then add this git repo to the client config file and restart. You can find examples and read more about this here.

March 4, 2021 at 11:44pm
Cool, will try soon. The client config file is at ~/gigantum/.labmanager/config.yaml? I use the Gigantum client image on DigitalOcean all the time, it’s pretty awesome. One click, 30 seconds later I can work on ML models with huge data sets while being in the mountains with crap internet. But, it seems like not all base envs are available that can be found here https://github.com/gigantum/base-images
gigantum + ipad + magic keyboard although on 3g is fire
like-fill
1

March 5, 2021 at 4:11pm
Great to hear you're finding the DO 1-click app useful!
We need to finish writing up documentation on the structure of that repo and how to roll your own bases. It's a common question we get a lot.
Some high-level tips that may make it easier to parse what's going on:
  • Each folder represents a base and all of its revisions. The UI will only display/let you select the latest base when creating a Project, but all bases still "exist" so older projects still build.
  • the _templates folder contains the dockerfile for images that resuse a common base. For example all the CUDA images are the same but have a different nvidia container they start from.
  • To create your own base image, the simplest thing to look at is probably the data science quickstart. If you look at the Dockerfile it starts from the python3-minimal base and installs a bunch of packages. If you do this, all the "gigantum specific stuff" like running jupyter on the right port, setting env vars that are expected, are handled.
  • You can fork this repo and make changes OR simply create a public git repo that contains a gigantum.yaml file, a folder with the name of your base, and a metadata yaml file in that folder.
  • the ~/gigantum/.labmanager/config.yaml file is what you create/edit. It's basically an "override" file so it is merged with the default full file. You only need to specify what you want to change. It is read at start so you have to restart the client for changes to take place.
  • the environment.repo_url field in the config file is a list, so you can replace ours, or add yours and both will be available like this:
environment:
repo_url:
- "https://github.com/gigantum/base-images.git"
- "https://github.com/<your-github-namespace>/base-images.git"
iframe:
enabled: false
allowed_origin: localhost:10000