Ubiquitous computing is extending its applications to an increasing number of new domains. The implementation of most of the newly appeared scenarios needs to follow guidelines coming from an expert of Interaction Design (IxD). In this paper we present a work-in-progress framework to conduct human-computer and computer-computer interactions and a tool for IxD applying it, as case studies, both in home and museum scenarios. Our framework uses multi-modal interaction and smart-devices, the main computational effort is dedicated to the design of a distributed chain of control monitoring sensors, devices and humans in the environment. The framework aims at merging concepts from the Intelligent Environments with those coming from Natural User Interaction theories. Moreover, the system includes designing procedures to support developers and users to produce new application just working on higher level layers separated from the infrastructural technologies. Preliminary results, coming from tests in Wizard of Oz, highlight potentialities of our framework.