PhD Position F/M Anticipation of Hand-Object Contact Configuration for Object Manipulation

PhD Position               ·               Montbonnot              ·       Full time

PhD Position F/M Anticipation of Hand-Object Contact Configuration for Object Manipulation

Job offer in pdf format

Contract type : Fixed-term contract

Level of qualifications required : PhD or equivalent

Fonction : PhD Position

About the research centre or Inria department

The Inria Grenoble research center groups together almost 600 people in 23 research teams and 7 research support departments.

Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (University Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.

Inria Grenoble is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.

Context

Collaboration between Inria team THOTH and the multidisciplinary institute in artificial intelligence (MIAI) in Grenoble

Research activities in MIAI aim to cover all aspects of AI and applications of AI with a current focus on embedded and hardware architectures for AI, learning and reasoning, perception and interaction, AI & society, AI for health, AI for environment & energy, and AI for industry 4.0.

This project in particular focuses on perception and robot interaction and thus will take place in close collaboration with LIG and GIPSA-Lab at the University of Grenoble.

Assignment

Key words: deep learning, trajectory prediction, manipulation, contact

Theme/Domain: Anticipation of Hand-Object Contact Configuration for Object Manipulation 

Context and Motivation

Learning hand-object contact configurations typically use large data sets consisting of 3D object shapes together with hand (robot gripper) poses, that together determine stable object-hand configurations. Still most approaches require complicated refinement strategies that disregard infeasible hand-object configurations from physically plausible ones [1]. In this work package we propose to jointly model both, the manipulation strategy together with the final object-hand configuration that should be reached. The intuition behind this approach is that knowledge about the appropriate action (e.g. manipulation strategy – sliding, picking) for a specific scenario determines the prediction of final object-hand configurations. Thus unrealistic configurations that are either not reachable or interfere with the environment in a physically not plausible won’t be even raised as a possibly feasible option.

Contact

pia.bideau@inria.fr

Please apply with the typical application documents: CV, certificates (MSc degree), research statement and two references. Application deadline is November 30th. We will screen applications on a rolling basis. The position will be filled as soon as a suitable candidate is found.

Main location: INRIA Montbonnot, TOTH team

Main activities

Summary

The knowledge about possible manipulation strategies and the object lead to feasible hand-object configurations. This projects aims at determining feasible hand-object contact configurations for object manipulation.

Approach

Many learning based approaches that consider only the object and the manipulator, generate hand-object configurations that are either not reachable or interfere with the environment in an harmful manner. For realisation, we will capitalise on the recent trend of action prediction using conditional variational autoencoders (CVAE) [2,3]. CVAEs are versatile deep generative that extend the standard VAE by conditioning with auxiliary properties – thus are suitable to develop visual representations conditioned on activity information. We will follow-up on existing concepts that will be further developed:

• Learning a (latent) representation conditioned on high-level manipulation strategies.
• Semi-supervised learning for learning from just few (but correct) examples. This stands in contrast to learning from massive amounts of noisy data.

Goal

Developing representations that jointly model visual information and dynamic action understanding – motion and contact. These representations will allow fundamentally new ways of interpreting grasps as dynamic motion trajectories that are not only defined by pre-computed static hand-object configurations, further this new way of modelling hand-object configurations will allow for incorporating information about a suitable manipulation strategy. 

References

• Mousavian, A., Eppner, C., & Fox, D. (2019). 6-dof graspnet: Variational grasp generation for object manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2901-2910).
• Halawa, M., Hellwich, O., & Bideau, P. (2022, October). Action-based contrastive learning for trajectory prediction. In European Conference on Computer Vision (pp. 143-159). Cham: Springer Nature Switzerland.
• Noseworthy, M., Paul, R., Roy, S., Park, D., & Roy, N. (2020, May). Task-conditioned variational autoencoders for learning movement primitives. In Conference on robot learning (pp. 933-944). PMLR.

Skills

• MSc degree in computer science or similar field
• Experience in robotics, machine learning, computer vision, and/or control, industrial experience is a plus.
• Experience in applying (deep) learning to robotic control problems
• Excellent software engineering and programming skills in C++ and/or Python
• Interest in interdisciplinary research in the context of the MIAI Grenoble Alpes Institute
• Excellent English writing and communication skills

Benefits package

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking (90 days / year) and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage under conditions

Remuneration

1st and 2nd year: 2 082 euros gross salary /month
3rd year: 2 190 euros gross salary / month

Apply for this position 

General Information

• Theme/Domain : Robotics and Smart environments Information system (BAP E)
• Town/city : Montbonnot
• Inria Center : Centre Inria de l’Université Grenoble Alpes 
• Starting date : 2023-03-01
• Duration of contract : 3 years
• Deadline to apply : 2023-11-30

Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.

Instruction to apply

Applications must be submitted online on the Inria website.

Processing of applications sent by other channels is not guaranteed.

Defence Security :

This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.

Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.

Contacts

• Inria Team : THOTH 
• PhD Supervisor :
  Bideau Pia / bideau@inria.fr

About Inria

Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.