Needle insertion simulators find use in a number of medical interventions, such as prostate brachytherapy. A needle insertion simulator has three main components: the needle model, the tissue model, and the model of interaction between the needle and the tissue. In this paper, a new methodology is introduced for the joint modeling of tissue and needle-tissue interactions. The approach consists of the measurement of tissue motion using ultrasound, and of the needle position and base force. Tissue motion is determined using a correlation-based algorithm that processes the ultrasound radiofrequency data. The tissue elastic parameters and the parameters of the tissue-needle interaction model are determined by using numerical optimization to match the response of the needle insertion model to the measured data. Phantom experiments were carried out in which a brachytherapy needle was inserted into a two-layer non-homogeneous phantom mimicking a prostate and its surrounding tissue. Experimental results show good agreement with the model obtained. In particular, the parameters of a three-parameter force model were identified for each layer of the phantom to fit the measured force to the simulated one. Also, the Young's modulus of each layer was identified to match the measured and simulated nodal axial displacements. This is the first report of the use of ultrasound radiofrequency data to characterize tissue motion during needle insertion. As the method is non-invasive and does not involve ionizing radiation, its application in patient studies is feasible.