We consider the problem of identifying switched linear state space models from a finite set of input-output data. This is a challenging problem, which requires inferring both the discrete state and the parameter matrices associated with each discrete state. An important contribution of our work is that we do not make the restrictive assumption of minimum dwell time between the switches, as it is customary in methods that deal with such models. We first propose a technique for eliminating the unknown continuous state from the model equations under an appropriate assumption of observability. On a time horizon, this gives us a new switched input-output relation that involves structured intermediary matrices, which depend on the state space representation matrices. To estimate the intermediary matrices, we present a randomly initialized algorithm that alternates between data classification and parameter update via recursive least squares. Given these matrices, the parameters associated to the different discrete states can be computed after a correct estimation of the discrete state.