snorkel

Spam Tutorial: Baseline models perform better than model with snorkel labels

I'm just getting started with Snorkel and working through the spam tutorial here: https://github.com/snorkel-team/snorkel-tutorials/blob/master/spam/01_spam_tutorial.ipynb If I set all the proper seeds, I can reproduce most of the results. However towards the end, when we…

Has anyone tried doing NER with Snorkel? If so, can you provide an example?

Hi - newbie here (I'm a software engineer @Google).

I'm having trouble following the Get Started tutorial. Specifically, after 'pip install snorkel', "from utils import load_unlabeled_spam_dataset" still fails. Is there another Python package to install?

multi-class labeling

Hello, all! I am wondering if the LF allows the rules for defining multiple classes? for example: Rule-> class A and class B or Rule -> class A or Class B. Also, if LabelModel allows such LFs. if not, what's the best practice. Thanks!

snorkel tutorial for NER

Thanks @braden-hancock for directing me to the right channel for this request. Would like to propose new tutorial related to NER to be added in the coming new release of snorkel.

Understanding buckets key in error_analysis.py

In error_analysis.py class documentation for get_label_buckets method, it's mentioned that "The returned buckets[(i, j)] is a NumPy array of data point indices with predicted label i and true label j." Don't you think that i and j are true labels and predicted labels,…

I am confused: What is the role of labeled data in Snorkel? Thanks!

When I first started reading about Snorkel I had the impression that it could learn to label without any labeled data whatsoever, which seems amazing, if not impossible -- almost like a perpetual motion machine: "We show...we can recover accuracies...without any labeled…

Relationship between MeTal and Snorkel?

Hi, I'm just starting to learn about Snorkel and have also been reading about the MeTaL work. How does MeTaL differ from Snorkel? I'm giving a talk on Snorkel on Friday, so it'd help me to know! Is MeTaL just a precursor to Snorkel that is now subsumed by it? Thanks! -- Bill

Spouse relation extraction performance

This paper (https://arxiv.org/pdf/1711.10160.pdf) shows discriminator F1 to be 54.2 but at the very end of the spouses demo (https://github.com/snorkel-team/snorkel-tutorials/blob/master/spouse/spouse_demo.ipynb), F1 score obtained is 28.63. Why is this difference? I initially…

Multitask learning tutorial

I would like to see an example of hierarchical multitask learning