Cristina

CRISTINA SEGALIN

Computer Vision Senior Research Scientist

I am a senior research scientist in Machine Learning and Computer Vision. I work on Social Signal Processing, Affective Computing and Creative AI.

Email: csegalin@netflix.com, segalin.cristina@gmail.com
RESUMÉ | LINKEDIN | GOOGLE SCHOLAR |TWITTER | GITHUB

RESEARCH INTERESTS
My interest is at the intersection of Machine Learning, Computer Vision, Machine Perception, Human Sciences, Social Signal Processing, Affective Computing and Creative/Generative AI. I am interested in the potential of AI systems to develop new forms and processes for human creativity to use them as non-human collaborators and empower creative expression. I am also interested in building integrated systems that can sense human (social, verbal and non-verbal) behavior in order to understand, model and synthesize social interactions and affect to provide computers with similar abilities.
Other areas I work on: Computational Aesthetics, Social Media Analysis, Object Detection/Recognition, Pose Estimation, Action Recognition, Biometrics, Re-Identification, Neuroscience, Computational Ethology, Human-Computer Interaction, Virtual/Augmented Reality.

SHORT BIO
BSc in Multimedia Computer Science (2010), MSc in Engineering and Computer Science (2012) and PhD in Computer Science (2016) at the Department of Computer Science of the University of Verona (Italy). During my PhD I investigated the interplay between aesthetic preferences and individual differences. Research associate at Disney Research (2016). Postdoctoral scholar at CalTech (2016-2018) under the supervision of Pietro Perona, where I worked on the analysis, detection, tracking and recognition of mice social behaviors in videos.
Research scientist at Disney Research LA (2018-2020) where I built integrated systems that can sense human (social, verbal and non-verbal) behavior in order to deliver more seamless experiences. I developed applications for understanding, modeling and synthesizing social interactions to provide computers with similar abilities.
In 2020 I joined Netflix as CV senior research scientist, where I am part of the Studio Media Algo DSE team. Here, I research and develop algorithms, models and pipelines at scale to enhance the tools used by content creators throughout the production lifecycle, including visual effects (VFX), animation and games. I deploy algorithms that assist and empower editors, creatives and artists in their daily workflows in generating media assets and original content.

PUBLICATIONS
 

Disclaimer. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All people copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

OTHER PROJECTS
DATASET & CODE
 

 

 

A New Soft Biometric Trait: Favorite Images

This dataset is used for studying personal aesthetics, a recent soft biometrics application where the goal is to recognize people from the images they like. It's composed of 200 users, 40K images. Given a set of preferred image of a user.
[ACCV14] [ICMI14] [IEEEForensics14] [ICIP14]

[Dataset]

 

 

Personality and Images: PsychoFlickr

This dataset is used to infer both self-assessed and attributed personality traits (Big-Five Traits) of Flickr users from their galleries of favorite pictures. The datset is composed of 60,000 pictures tagged as favorite by 300 users.
[CVIU16] [IEEEAC16] [ACMBNI13]

[Dataset] [Code Features]

 

 

 

Mice Action Recognition System (MARS)

Software to analyze mice social interactions in videos. Deep Learning mice detector, pose estimation, tracking and behavior classifier in Tensorflow. The system is optimized and integrated into a GUI.
[ELIFE21]

[Code]

 

 

Predicting the personal appeal of marketing images

“How appealing is this image to this particular consumer?” Using the five‐factor model of personality, we predict an image's personality appeal — the personality of consumers to which the image appeals most.
[JCP19]

[Code]

ABOUT
 

I attended a bachelor degree in Multimedia Information Technology at the University of Verona, where I was mainly interested in Computer Vision, Augmented Reality, Human Computer Interaction, Computer Graphics and also Perception. My bachelor degree thesis proposed a face recognition system, that has been installed at the door of the VIPS laboratory of the University of Verona. I completed a master degree on Engineering and Computer Science with a thesis focused on the research field of Social Signal Processing (SSP), the domain aimed at modeling, analysis and synthesis of nonverbal communication in human-human and human-machine interactions. The purpose of the thesis was the person re-identfication through the way people chat with other subjects. During that period, I was lucky enough to work also on other research projects, like recognizing the age of children by the way they talk with each other. SSP together with, Social Media Analysis, Personality Computing, Machine Learning and Computer Vision became the main topics of my PhD at the Dept. of Computer Science in Verona (Italy).

During the first year of PhD I investigated the interplay between aesthetic preferences and individual differences, under the supervision of Marco Cristani. I had the great opportunity to move to Glasgow for some months and collaborate with Alessandro Vinciarelli to this project. I collected a dataset of 60K images favorited from Flickr users, extracted features coming from the field of Computational Aesthetics (CA), and predicted from them the personality of a user. Continuing on the perspective of CA, we also proposed a soft biometrics application where the goal is to recognize people by considering the images they like as a new biometric trait. At the end of the second year of PhD I moved for some months in Birmingham to collaborate with Mirco Musolesi with the aim of investigating the role of textual, visual and social cues in information propagation in Twitter. My last contribute during the PhD was in the field of Deep Learning and Representation Learning, trying generalize the particular cues that characterize each personality trait. While waiting to defend my Phd thesis, I worked as research associate at Disney Research in Pittsburgh (PA).

After my graduation, I moved to Pasadena (CA) to work as postdoctoral scholar at California Institute of Technology (Caltech), in the Computational Vision Lab, under the supervision of Pietro Perona and to collaborate with David Anderson. Here I worked on animal behavior, in particular, Computational Ethology, which involves biologists and computer scientists with the common goal of understanding, analyzing, measuring and describing animal behavior using Machine Learning and Machine Vision algorithms and tools. My role here was to develop a novel system able to detect, track and recognize mice actions on videos.

Between 2018 and 2020 I was a research scientist at Disney Research LA, working on Machine Learning, Computer Vision, Creative AI, Emotional AI and Affective Computing and Perception with the goal of creating new magic experiences in the theme parks, resorts, hotels and cruiseships. I build integreated systems that can sense human (social and non-verbal) behavior in order to deliver more seamless experiences. In particular I developed applications for understanding, modeling and synthetizing social interactions to provide computers with similar abilities. I also explored the potential of AI system to develop new forms of and processes for human creativity in order to use them as non-human collaborators and empower creative expression.

In 2020 I joined Netflix as CV senior research scientist, where I am part of the Studio Media Algo DSE team. Here, I research and develop algorithms, models and pipelines at scale to enhance the tools used by content creators throughout the production lifecycle, including visual effects (VFX), animation and games. I deploy algorithms that assist and empower editors, creatives and artists in their daily workflows in generating media assets and original content.

Current Position

  • CV Senior Research Scientist, Netflix, Los Gatos, CA, USA (2020-)

 

Research and Work Experience

  • Research Scientist, Disney Research Los Angeles, Glendale, CA, USA (2018-2020).
  • Postdoctoral scholar, Computational Vision Lab, California Institute of Technology, Pasadena, CA, Advisor: P. Perona (2016-2018).
  • DR Internship, Dinsey Reseach, Pittsburgh, PA, Tutor: Maarten Bos (2016).
  • UoB Research Assistant, School of Computer Science, Birmingham, UK, Academic Tutor: M.Cristani, Tutor: M. Musolesi.
  • GLA.DCS Research Assistant, School of Computing Science, Glasgow, Scotland, Academic Tutor: M. Cristani, Tutor: A. Vinciarelli.
  • IIT Research Assistant, IIT Genova, Genova, Tutor: V. Murino.
  • VIPS Lab Research Assistant, Computer Science Dept., University of Verona, Academic Tutor: M. Cristani, Tutor: U. Castellani.
  • VIPS Lab Research Assistant, Computer Science Dept., University of Verona, Academic Tutor: U. Castellani, Tutor: M. Cristani.

 

Education

  • PhD in Computer Science, Dept. of Computer Science, University of Verona, Advisor: M. Cristani
  • Thesis title: A Social Signal Processing Perspective on Computational Aesthetics: Theories and Applications.
  • Thesis (pdf, slides)
  • Master Degree in Computer Engineering and Computer Science, Curriculum in Visual Computing, University of Verona.
  • Thesis title: Statistical analysis of Skype conversations: recognizing individual by their chatting style. Advisor: M. Cristani
  • Thesis (pdf)
  • Bachelor Degree in Multimedia Information Technology, University of Verona
  • Thesis title: Sistema di rilevamento automatico e riconoscimento volti: aspetti metodologici e pratici. Advisor: U.Castellani
  • Thesis (pdf)
  • High School Diploma in Ragioneria and expert programmer.

 

Awards and Grants

  • ICVSS Best Presentation Award 2015 - International Computer Vision Summer School best poster presentation.
  • Erasmus Placement 2013 - Second over four grants assigned to PhD students for placements in companies, facilities.
  • PhD Scholarship - University of Verona that support my PhD from January 2013 to December 2015.

 

Reference List

  • Pietro Perona - California Institute of Technology, Pasadena, CA. Email: perona@caltech.edu
  • Marco Cristani - University of Verona, Verona, Italy. Email: marco.cristani@univr.it
  • Alessandro Vinciarelli - School of Computing Science, University of Glasgow (Glasgow) and Idiap (Switzerland).
    Email: vincia@dcs.gla.ac.uk

 

Given Talks

  • Dec 2020 - Computer Vision at Netflix - NeurIPS - Virtual
  • Feb 2018 - Social Profiling through Image Understanding - Legendary Analytics - Boston, MA (USA)
  • Feb 2018 - Unveiling the Multimedia Unconscious - MIT Media Lab - Boston, MA (USA)
  • Feb 2018 - Social Profiling through Image Understanding - MIT Media Lab - Boston, MA (USA)
  • Oct 2017 - What your Facebook Profile Picture Reveals about your Personality - ACM MM, Mountain View, CA (USA)
  • May 2017 - Social Profiling through Image Understanding - Association for Psychological Science Convention - Boston, MA (USA)
  • Dec 2016 - Social Profiling through Image Understanding - NIPS, Barcelona (ES)
  • Feb 2016 - Computational Aesthetics for Multimedia, a Social Signal Processing Perspective - Caltech, Pasadena, CA (USA)
  • Oct 2015 - Computational Aesthetics for Multimedia - UNSW School of Psychology, Sydney (AU)
  • Oct 2015 - Computational Aesthetics for Multimedia - UTS, Sydney (AU)
  • Oct 2015 - Computational Aesthetics for Multimedia - NICTA, Canberra (AU)
  • July 2015 - ICVSS oral presentation for best poster presentation award - Sicily (IT)
  • Oct 2014 - Deep Learning Applications - Dept. of Computer Science, University of Verona, Verona (IT)
  • July 2014 - Introduction to Deep Learning - Dept. of Computer Science, University of Verona, Verona (IT)

 

Reviewer

  • Conferences: ACCV, ICCV, CVPR, ECCV, NeurIPS, CHI, ICML, ICLR, CVUE
  • Journal: PLOS ONE, NEUROCOM, IEEE Multimedia, Affective Computing, AIMed, IJCV

 

Program Committee

  • 3rd Workshop on Media Analytics for Societal Trends at ACM MM
  • 2rd Workshop on Media Analytics for Societal Trends at ACM MM

 

Organizer EMERGENT workshop at ACII 2019

  • Lead the effort to organize the first workshop on Emotions and Emergent States in Groups.
  • Some of the top affective computing scientists attended the workshop.

 

Workshop mentor

  • Women in Machine Learning, NeurIPS 2019
  • Women in Computer Vision, CVPR 2019
  • Women in Machine Learning, NeurIPS 2018

 

Research Mentor

  • Philine Witzig, Bryan Loh - DRLA, CA
  • Zack Polizzi, David Mace, Jennifer Sun - CalTech, Pasadena, CA
  • Francesca Zerbato, Luca Brunelli, Marco Fanini, Elena Boschetti - University of Verona, Verona, Italy - Master's Thesis

 

Media Coverage

 

SW / HW

  • Coding: Python, C++, C\#, R, Matlab, SQL /li>
  • ML Tools:PyTorch, Tensorflow, Keras, SciPy, Scikit, OpenCV, Pandas, Ray
  • Creative Tools: Adobe Creative Suite, DaVinci, Automatic1111, Midjourney
  • Framework: Metaflow, AWS (S3, EC2, SageMaker)
  • Web: HTML5, CSS, JavaScript, NodeJS, Flask, Gradio
  • Misc: Git, LaTeX, ARToolkit, Flickr API, Twitter API, Google API, LucidChart
  • OS: Windows, Linux, Mac OS, ROS

    •  

    Lead

  • Mentor, Project Lead & Researcher, Hands on experience conducting research and leading projects to develop, optimize, deploy and ship to production algorithms and systems. Set technical and creative direction to pursue +2 years projects. Guided multidisciplinary researchers and engineers

    •  

    Technical

  • Qualitative & Quantitative Researcher, Rapid Prototyping, Academic Writing, Software and System Developer, Computer Vision, Machine Learning, Deep Learning (CNN, RNN, GAN, VAE, Transformers, Diffusion Models), Image Processing, Data Science, Infrastructure/System dDsign and Optimization, Data Annotation Pipelines, Web/UI/UX Design and Development.

    •  

    Soft

  • Dynamic Researcher, Inclined to analyze both theoretical and practical problems, solving them by employing innovative and creative thinking, rapid prototyping. Effective collaboration in cross-functional teams, strong communication skills. Strong abilities to blend in new contexts, maintain productivity and quality standards while managing multiple projects concurrently, agility in transitioning between projects, efficiently allocating resources, adjusting strategies and effectively managaing shifting priorities. Eager to learn and develop new skills.

    •  

    Personal

  • Driving License: B
  • Hobby and sports: Travelling, reading, cooking, watching TV shows, Yoga, Running, Muay Thai, Lifting, eating sushi and a lot of chocolate!!

    •  

    graphic