Autonomous systems requiring user supervision or manual control of autonomy are becoming more prevalent in real world deployments. These systems will require transitions of control between autonomous and manual operations with users being required to both take control from and cede control to autonomy. Prior work in Human-Drone Interaction (HDI) has observed or designed for user interaction with a perfectly functioning robot, but has not looked at interactions with a robot that is about to fail. In this paper we describe results from initial work on characterizing user responses to failures in aerial autonomous systems. Ongoing and future work involves evaluating user proficiency in system operation and its impact on HDI with semi-autonomous systems. This work is novel in the context of small Unmanned Aerial Vehicles (sUAVs) and will inform sUAV autonomy designers for systems with a range of user training from search and rescue to hobbyist users through recommendations for training, necessary timelines for information sharing, and failure planning or contingency options in HDI.