Cristina

CRISTINA SEGALIN

Computer Vision Senior Research Scientist · Netflix_logo

I am a Senior Research Scientist in Machine Learning and Computer Vision. I work on Social Signal Processing, Affective Computing and Creative AI.

Email: csegalin@netflix.com, segalin.cristina@gmail.com
RESUMÉ | LINKEDIN | GOOGLE SCHOLAR |TWITTER | GITHUB | ORCID

RESEARCH INTERESTS
Machine Learning, Deep Learning, Computer Vision (Object Detection/Recognition, Pose Estimation, Action Recognition), Creative AI, Machine Perception, Human-Computer Interaction, Human Sciences, Social Signal Processing, Affective Computing, Social Media Analysis, Image/Video Processing, Computational Aesthetics, Biometrics, Neuroscience, Computational Ethology, Re-Identification, Biometrics, Virtual/Augmented Reality, Brain Computer Interfaces.

SHORT BIO
BSc in Multimedia Computer Science (2010), MSc in Engineering and Computer Science (2012) and PhD in Computer Science (2016) at the Department of Computer Science of the University of Verona (Italy). During my PhD I investigated the interplay between aesthetic preferences and individual differences. Research associate at Disney Research (2016). Postdoctoral scholar at CalTech (2016-2018) where I worked on the analysis, detection, tracking and recognition of mice social behaviors in videos. Research scientist at Disney Research LA (2018-2020) where I built integreated systems that can sense human (social, verbal and non-verbal) behavior in order to deliver more seamless experiences. I developed applications for understanding, modeling and synthetizing social interactions to provide computers with similar abilities. I also explored the potential of AI system to develop new forms of and processes for human creativity in order to use them as non-human collaborators and empower creative expression.
In 2020 I joined Netflix as CV senior research scientist, where I am part of the AE-CAPE (Algorithm Evidence - Creative Algorithms and Personalization Engineering) team, which is responsible for algorithmic creation and personalization of Netflix movies, TV shows, and promotional media assets. I develop Computer Vision and Machine Learning algorithms to analyze and transform raw media sources to generate and recommend media assets, such as artwork and video trailers that introduce the Netflix content to our 200+ million members. Cross-functional work includes research, design, implementation, A/B testing, and deploying of algorithms into production. Other areas of research include algorithmically assisting the post-production of our original content.

PUBLICATIONS

Disclaimer. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All people copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

OTHER PROJECTS
DATASET & CODE

A New Soft Biometric Trait: Favorite Images

This dataset is used for studying personal aesthetics, a recent soft biometrics application where the goal is to recognize people from the images they like. It's composed of 200 users, 40K images. Given a set of preferred image of a user.
[ACCV14] [ICMI14] [IEEEForensics14] [ICIP14]

[Dataset]

Personality and Images: PsychoFlickr

This dataset is used to infer both self-assessed and attributed personality traits (Big-Five Traits) of Flickr users from their galleries of favorite pictures. The datset is composed of 60,000 pictures tagged as favorite by 300 users.
[CVIU16] [IEEEAC16] [ACMBNI13]

[Dataset] [Code Features]


Mice Action Recognition System (MARS)

Software to analyze mice social interactions in videos. Deep Learning mice detector, pose estimation, tracking and behavior classifier in Tensorflow. The system is optimized and integrated into a GUI.
[In preparation]

[Code (coming soon)]

Predicting the personal appeal of marketing images

“How appealing is this image to this particular consumer?” Using the five‐factor model of personality, we predict an image's personality appeal — the personality of consumers to which the image appeals most.
[JCP19]

[Code]

ABOUT

I attended a bachelor degree in Multimedia Information Technology at the University of Verona, where I was mainly interested in Computer Vision, Augmented Reality, Human Computer Interaction, Computer Graphics and also Perception. My bachelor degree thesis proposed a face recognition system, that has been installed at the door of the VIPS laboratory of the University of Verona. I completed a master degree on Engineering and Computer Science with a thesis focused on the research field of Social Signal Processing (SSP), the domain aimed at modeling, analysis and synthesis of nonverbal communication in human-human and human-machine interactions. The purpose of the thesis was the person re-identfication through the way people chat with other subjects. During that period, I was lucky enough to work also on other research projects, like recognizing the age of children by the way they talk with each other. SSP together with, Social Media Analysis, Personality Computing, Machine Learning and Computer Vision became the main topics of my PhD at the Dept. of Computer Science in Verona (Italy).

During the first year of PhD I investigated the interplay between aesthetic preferences and individual differences, under the supervision of Marco Cristani. I had the great opportunity to move to Glasgow for some months and collaborate with Alessandro Vinciarelli to this project. I collected a dataset of 60K images favorited from Flickr users, extracted features coming from the field of Computational Aesthetics (CA), and predicted from them the personality of a user. Continuing on the perspective of CA, we also proposed a soft biometrics application where the goal is to recognize people by considering the images they like as a new biometric trait. At the end of the second year of PhD I moved for some months in Birmingham to collaborate with Mirco Musolesi with the aim of investigating the role of textual, visual and social cues in information propagation in Twitter. My last contribute during the PhD was in the field of Deep Learning and Representation Learning, trying generalize the particular cues that characterize each personality trait. While waiting to defend my Phd thesis, I worked as research associate at Disney Research in Pittsburgh (PA).

After my graduation, I moved to Pasadena (CA) to work as postdoctoral scholar at California Institute of Technology (Caltech), in the Computational Vision Lab, under the supervision of Pietro Perona and to collaborate with David Anderson. Here I worked on animal behavior, in particular, Computational Ethology, which involves biologists and computer scientists with the common goal of understanding, analyzing, measuring and describing animal behavior using Machine Learning and Machine Vision algorithms and tools. My role here was to develop a novel system able to detect, track and recognize mice actions on videos.

Between 2018 and 2020 I was a research scientist at Disney Research LA, working on Machine Learning, Computer Vision, Creative AI, Emotional AI and Affective Computing and Perception with the goal of creating new magic experiences in the theme parks, resorts, hotels and cruiseships. I build integreated systems that can sense human (social and non-verbal) behavior in order to deliver more seamless experiences. In particular I developed applications for understanding, modeling and synthetizing social interactions to provide computers with similar abilities. I also explored the potential of AI system to develop new forms of and processes for human creativity in order to use them as non-human collaborators and empower creative expression.

In 2020 I joined Netflix as CV senior research scientist, where I am part of the AE-Evidence team, which is responsible for algorithmic creation and personalization of Netflix media assets. I develop Computer Vision and Machine Learning algorithms to analyze and transform raw media sources to generate and recommend media assets, such as artwork and video trailers that introduce the Netflix content to our 190+ million members. Cross-functional work includes research, design, implementation, A/B testing, and deploying of algorithms into production. Other areas of research include algorithmically assisting the post-production of our original content.

Current Position

  • CV Senior Research Scientist, Netflix, Los Gatos, CA, USA (2020-)
  • Research and Work Experience

  • Research Scientist, Disney Research Los Angeles, Glendale, CA, USA (2018-2020).
  • Postdoctoral scholar, Computational Vision Lab, California Institute of Technology, Pasadena, CA, Advisor: P. Perona (2016-2018).
  • DR Internship, Dinsey Reseach, Pittsburgh, PA, Tutor: Maarten Bos (2016).
  • UoB Research Assistant, School of Computer Science, Birmingham, UK, Academic Tutor: M.Cristani, Tutor: M. Musolesi.
  • GLA.DCS Research Assistant, School of Computing Science, Glasgow, Scotland, Academic Tutor: M. Cristani, Tutor: A. Vinciarelli.
  • IIT Research Assistant, IIT Genova, Genova, Tutor: V. Murino.
  • VIPS Lab Research Assistant, Computer Science Dept., University of Verona, Academic Tutor: M. Cristani, Tutor: U. Castellani.
  • VIPS Lab Research Assistant, Computer Science Dept., University of Verona, Academic Tutor: U. Castellani, Tutor: M. Cristani.
  • Education

  • PhD in Computer Science, Dept. of Computer Science, University of Verona, Advisor: M. Cristani
    • Thesis title: A Social Signal Processing Perspective on Computational Aesthetics: Theories and Applications.
    • Thesis (pdf, slides)
  • Master Degree in Computer Engineering and Computer Science, Curriculum in Visual Computing, University of Verona.
    • Thesis title: Statistical analysis of Skype conversations: recognizing individual by their chatting style. Advisor: M. Cristani
    • Thesis (pdf)
  • Bachelor Degree in Multimedia Information Technology, University of Verona
    • Thesis title: Sistema di rilevamento automatico e riconoscimento volti: aspetti metodologici e pratici. Advisor: U.Castellani
    • Thesis (pdf)
  • High School Diploma in Ragioneria and expert programmer.
  • Awards and Grants

  • ICVSS Best Presentation Award 2015 - International Computer Vision Summer School best poster presentation.
  • Erasmus Placement 2013 - Second over four grants assigned to PhD students for placements in companies, facilities.
  • PhD Scholarship - University of Verona that support my PhD from January 2013 to December 2015.
  • Reference List

  • Pietro Perona - California Institute of Technology, Pasadena, CA. Email: perona@caltech.edu
  • Marco Cristani - University of Verona, Verona, Italy. Email: marco.cristani@univr.it
  • Alessandro Vinciarelli - School of Computing Science, University of Glasgow (Glasgow) and Idiap (Switzerland).
    Email: vincia@dcs.gla.ac.uk
  • Given Talks

  • Feb 2018 - Social Profiling through Image Understanding - Legendary Analytics - Boston, MA (USA)
  • Feb 2018 - Unveiling the Multimedia Unconscious - MIT Media Lab - Boston, MA (USA)
  • Feb 2018 - Social Profiling through Image Understanding - MIT Media Lab - Boston, MA (USA)
  • Oct 2017 - What your Facebook Profile Picture Reveals about your Personality - ACM MM, Mountain View, CA (USA)
  • May 2017 - Social Profiling through Image Understanding - Association for Psychological Science Convention - Boston, MA (USA)
  • Dec 2016 - Social Profiling through Image Understanding - NIPS, Barcelona (ES)
  • Feb 2016 - Computational Aesthetics for Multimedia, a Social Signal Processing Perspective - Caltech, Pasadena, CA (USA)
  • Oct 2015 - Computational Aesthetics for Multimedia - UNSW School of Psychology, Sydney (AU)
  • Oct 2015 - Computational Aesthetics for Multimedia - UTS, Sydney (AU)
  • Oct 2015 - Computational Aesthetics for Multimedia - NICTA, Canberra (AU)
  • July 2015 - ICVSS oral presentation for best poster presentation award - Sicily (IT)
  • Oct 2014 - Deep Learning Applications - Dept. of Computer Science, University of Verona, Verona (IT)
  • July 2014 - Introduction to Deep Learning - Dept. of Computer Science, University of Verona, Verona (IT)
  • Reviewer

  • Conferences: ACCV, ICCV, CVPR, ECCV, NIPS, CHI, ICML, ICLR
  • Journal: PLOS ONE, NEUROCOM, IEEE Multimedia, Affective Computing, AIMed
  • Program Committee

  • 3rd Workshop on Media Analytics for Societal Trends at ACM MM
  • 2rd Workshop on Media Analytics for Societal Trends at ACM MM
  • Organizer EMERGENT workshop at ACII 2019

  • Lead the effort to organize the first workshop on Emotions and Emergent States in Groups.
  • Some of the top affective computing scientists attended the workshop.
  • Workshop mentor

  • Women in Machine Learning, NeurIPS 2019
  • Women in Computer Vision, CVPR 2019
  • Women in Machine Learning, NeurIPS 2018
  • Research Mentor

  • Philine Witzig, Bryan Loh - DRLA, CA
  • Zack Polizzi, David Mace, Jennifer Sun - CalTech, Pasadena, CA
  • Francesca Zerbato, Luca Brunelli, Marco Fanini, Elena Boschetti - University of Verona, Verona, Italy - Master's Thesis
  • Media Coverage

  • 5 Lessons to Learn from a Disney Research Scientist - DMN
  • Disney’s Deep Dive on Personality Research, and Its Potential Implications for Brand Marketers - Street Fight
  • Social Profiling - Wikipedia
  • What machine vision means for creativity - Contagious
  • Computer Skills

  • Coding: Python, Matlab, Latex, HTML, CSS, C/C++/C#, SQL, Java, R
  • ML Tools:Tensorflow, Keras, PyTorch, SciPy, Scikit, OpenCV, Pandas
  • Web: HTML5, CSS, JavaScript, NodeJS, Flask
  • Tools: Git, LaTeX, ARToolkit, PRTools, Flickr API, Twitter API
  • OS: Windows, Linux, Mac OS, ROS
  • Multimedia/Design: Photoshop, Illustrator, Maya, Motion Builder
  • Qualities

  • Personal Skills: Inclined to analyze both theoretical and practical problems, able to solve them by employing novel concepts, good communication, eager to learn and develop new skills. Good software engineering abilities
  • Interpersonal Skills: Strong ability to blend in new contexts, good skills in teamwork, good organizational abilities, dynamism, desire for learning, extrovert. Project and team management.
  • Experience with: High level programming languages, digital design flow, validation and testing techniques,scripting languages, web interface development.
  • Driving License: B
  • Hobby and sports: Travelling, reading, cooking, watching TV shows, Yoga, Running, Muay Thai, eating sushi and a lot of chocolate!!
  • graphic