menu

neptune-community

A place where Neptune.ml users and developers come together to make things work

Channels
Team

diff for source code or parameters

August 4, 2019 at 8:14pm

August 4, 2019 at 8:17pm

Let me quote your request here:

  • reply
  • like
Hi, Kamil, I am missing to have a diff for source code or parameters (I have long dict with many dicts inside of dicts) between experiments to quickly see what actually changed it. Is it something already implemented?
  • reply
  • like

So, right now we have pretty fancy diffing for notebooks.

  • reply
  • like

Hey @jakub-czakon can you paste one example here? I remember that you have some good example of diffs between two notebooks.

Edited
  • reply
  • like

For now, you can diff parameters and code more manually, using source code tab and details tab respectively.

  • reply
  • like

Do you have any particular use case in mind, or it is your generic requirement @bezov ?

  • reply
  • like

Sure, I think this can be a good compare notebooks example

  • reply
  • like

August 5, 2019 at 3:48pm

My case looks as a general request. I have long data preparation pipeline with multiple tweaks to check (add new feature, change preprocessing, etc) along with the model parameters itself. Result controlled by long .json and the functions and the order of functions call in the code. Because parameters is multi-level dict - it is not readable in neptune details page and if I flatten it to one level - it becomes too long to manually compare. Same about code. Right now the only way is to compare git commits.

  • reply
  • like

Sure, I think this can be a good compare notebooks example

If I am not missing, the notebooks are not tight to the experiments? so its not an experiment diff and in order to make it I need to upload new notebook with new name regarding to every new experiment?

  • reply
  • like

@bezov notebooks are tied to experiments if you are using the extension. For example, see this project. There is a column Created in notebook and by clicking on it you go to the notebook version you need. It will bind it with the last checkpoint done from this particular notebook so just before you run your experiment I would suggest you click Upload.

  • reply
  • like

@bezov notebooks are tied to experiments if you are using the extension. For example, see this project. There is a column Created in notebook and by clicking on it you go to the notebook version you need. It will bind it with the last checkpoint done from this particular notebook so just before you run your experiment I would suggest you click Upload.

Thanks, Jakub, I overlooked this option, Will try.

  • reply
  • like

@bezov notebooks are tied to experiments if you are using the extension. For example, see this project. There is a column Created in notebook and by clicking on it you go to the notebook version you need. It will bind it with the last checkpoint done from this particular notebook so just before you run your experiment I would suggest you click Upload.

Jakub, by playing with notebooks I have next problem: by clicking on the "created in notebook" link next to the experiment get opened latest notebook checkpoint, but not the one associated with this experiment, so I have to guess by the date/time what checkpoint to choose for diff.

  • reply
  • like

August 6, 2019 at 12:51pm

Hi @bezov , right now experiments are linked to the notebook, not to the particular checkpoint.

  • reply
  • like

So, I consider it a feature reques, namely associate experiments with particular checkpoint.

  • reply
  • like