PhD Project Ideas

Here are some ideas for PhD that I could supervise. If you're interested, feel free to email me with questions about them or about what to do next.

phd_1 - Tractable situation coverage for autonomous cars

It is hard to test autonomous robot (AR) software because of the range and diversity of external situations (terrain, obstacles, humans, peer robots) that the robots must deal with. AR must interpret a wide variety of stimuli from their world and make decisions that are appropriate given the specific combination of stimuli they are receiving – their relationship with their environment is complex. Many features of the environment that matter are not specific to the mission goals – they are simply present, and must be handled. Beyond merely needing to avoid accidents, AR may need to comply with detailed-yet-ambiguous rules intended to guide human behaviour (such as the UK Highway Code and Rules of the Air).

Situation coverage [1] as a potentially important technique for testing AR. Put simply, situation coverage is a measure of the proportion of all possible situations (that the software under test could conceivably encounter) that have been tested by some test set. Like any coverage criterion, situation coverage can be used to assess the adequacy of a test set, and to guide automated test generation.

There are some existing models of situation spaces for 3D scenes [2]. To date, however, no-one has published a tractable and interesting situation space model for an autonomous vehicle.

In this PhD project, you could:

References

[1] Situation coverage - a coverage criterion for testing autonomous robots. Rob Alexander, Heather Hawkins, Drew Rae. Technical Report YCS-2015-496, Department of Computer Science, University of York, Jan 2015 https://www.cs.york.ac.uk/ftpdir/reports/2015/YCS/496/YCS-2015-496.pdf

[2] O. Zendel, W. Herzner, and M. Murschitz, ‘VITRO - Model based vision testing for robustness’, in 2013 44th International Symposium on Robotics (ISR), 2013, pp. 1–6. https://doi.org/10.1109/ISR.2013.6695667

phd_2 - Mutation testing for simulated autonomous vehicles

Mutation testing is an effective way to generate thorough tests for critical systems [1] and to evaluate testing methods [2]. It has been widely applied to many kinds of software, but not specifically to autonomous vehicles.

There is thus potential for a PhD project to produce a mutation testing approach to generate seeded faults in simulated autonomous vehicles:

References

[1] Y. Jia and M. Harman, ‘An Analysis and Survey of the Development of Mutation Testing’, IEEE Transactions on Software Engineering, vol. 37, no. 5, pp. 649–678, Sep. 2011. https://doi.org/10.1109/TSE.2010.62

[2] J. H. Andrews, L. C. Briand, and Y. Labiche, ‘Is Mutation an Appropriate Tool for Testing Experiments?’, in Proceedings of the 27th International Conference on Software Engineering, New York, NY, USA, 2005, pp. 402–411. https://doi.org/10.1145/1062455.1062530

phd_3 - A testbed simulator for autonomous vehicle testing methods

Prior work at York has developed an autonomous car simulator as part of a project to explore the value of a situation-coverage metric for robot testing (see [1] for an outline of the approach). The currently version of the simulator has a simulated robot with a number of seeded faults that can be switched on and off. This allows us to evaluate robot test generation approaches by applying them to the simulated robot and measuring how many of the seeded faults they find. This past work has weaknesses, however:

There is therefore potential for a PhD project thus:

After the project, the resulting taxonomy, simulator and experimental protocols will be of significant value to the autonomous vehicle development and test communities.

References:

[1] Situation coverage - a coverage criterion for testing autonomous robots Rob Alexander, Heather Hawkins, Drew Rae Technical Report YCS-2015-496, Department of Computer Science, University of York, Jan 2015 https://www.cs.york.ac.uk/ftpdir/reports/2015/YCS/496/YCS-2015-496.pdf

[2] Lussier B, Chatila R, Guiochet J, Ingrand F, Lampe A, Killijian M-O, et al. Fault Tolerance in Autonomous Systems: How and How Much. Proceedings of the 4th IARP/IEEE-RAS/EURON Joint Workshop on Technical Challenges for Dependable Robots in Human Environments, 2005. http://homepages.laas.fr/dpowell/documents/05250/05250.pdf

[3] Dependability and its Threats - a Taxonomy Algirdas Avizienis, Jean-Claude Laprie, Brian Randell http://rodin.cs.ncl.ac.uk/Publications/avizienis.pdf

[4] Fletcher L, Teller S, Olson E, Moore D, Kuwata Y, How J, et al. The MIT-Cornell Collision and Why it Happened. Journal of Field Robotics. 2008;25:775-807. http://onlinelibrary.wiley.com/doi/10.1002/rob.20266/pdf