Christopher Powers

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2020-87

May 28, 2020

Large labeled datasets are vital for applying supervised learning to computer vision tasks, but making these datasets takes time and effort. Human-machine collaboration has the potential to mitigate this cost. This work introduces a general-purpose framework for human-to-human and human-machine collaboration on image data. We show that by treating machine learning models as virtual users, multi-user synchronization can support versatile human-machine interaction; in other words, all you need is sync. In order to achieve synchronization behavior that seems correct to users, while also maintaining real-time editing speeds and supporting undo-redo, we adapt operational transformation [8] to the image labeling context. An open- source implementation of the collaboration system is presented as an in-progress addition to Scalabel, an annotation tool for visual data that was used to create the BDD100K dataset [24]. Finally, we give an example of how the collaboration feature can improve the labeling process by integrating Polygon-RNN++ [2] with Scalabel.

Advisors: Trevor Darrell

---

BibTeX citation:

@mastersthesis{Powers:EECS-2020-87,
    Author= {Powers, Christopher},
    Title= {A Scalable Synchronous System for Robust Real Time Human-Machine Collaboration},
    School= {EECS Department, University of California, Berkeley},
    Year= {2020},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-87.html},
    Number= {UCB/EECS-2020-87},
    Abstract= {Large labeled datasets are vital for applying supervised learning to computer vision tasks, but making these datasets takes time and effort. Human-machine collaboration has the potential to mitigate this cost. This work introduces a general-purpose framework for human-to-human and human-machine collaboration on image data. We show that by treating machine learning models as virtual users, multi-user synchronization can support versatile human-machine interaction; in other words, all you need is sync. In order to achieve synchronization behavior that seems correct to users, while also maintaining real-time editing speeds and supporting undo-redo, we adapt operational transformation [8] to the image labeling context. An open- source implementation of the collaboration system is presented as an in-progress addition to Scalabel, an annotation tool for visual data that was used to create the BDD100K dataset [24]. Finally, we give an example of how the collaboration feature can improve the labeling process by integrating Polygon-RNN++ [2] with Scalabel.},
}

EndNote citation:

%0 Thesis
%A Powers, Christopher 
%T A Scalable Synchronous System for Robust Real Time Human-Machine Collaboration
%I EECS Department, University of California, Berkeley
%D 2020
%8 May 28
%@ UCB/EECS-2020-87
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-87.html
%F Powers:EECS-2020-87