Autonomous driving technologies have been emerging over the past few years, and semi-autonomous driving functionalities have been deployed to vehicles available in the market. Since autonomous driving is realized by the intelligent processing of data from various types of sensors such as LIDAR, radar, camera, etc., the complexity of designing a dependable real-time autonomous driving system is rather high. Although there has been much research on building a reliable real-time system using hardware replication, the resulting systems tend to add significant extra cost due to hardware replication. Therefore, an alternative solution would be helpful in building an autonomous vehicle in a cost-effective way. An autonomous driving system is different from the conventional reliable real-time system because it requires (1) flexible design, (2) adaptive graceful degradation and (3) effective use of different modalities of sensors and actuators. To address these characteristics, we summarize SAFER (System-level Architecture for Failure Evasion in Real-time applications) our previous work on flexible system design. We then present a conceptual framework for autonomous vehicles to provide adaptive graceful degradation and support for using different types of sensors/actuators when a failure happens. We motivate our proposed framework with various scenarios, and we describe how SAFER can be extended to support the proposed conceptual framework.