Many existing algorithms for solving the vehicle routing problem with time windows (VRPTW) first construct initial tours and then apply a tour optimization algorithm to refine the solution. In this two-stage approach, the tour optimization stage is often hampered by the tour construction phase that produce initial solutions that are skewed, namely, the initial tours are very good, but the later tours are often very poor. This often leads to difficulties in the tour-optimization stage that often get trapped in local optimal quickly. In this paper, we propose a new multi-agent algorithm for solving the VRPTW that involves the uses a distributed, multi agent approach for the tour-optimization phase. Our approach can be considered as a combination of multi-agent system and heuristic local search. A prototype system has been developed and extensive experimentation on the Solomon benchmarks show that our multi-agent approach is effective and has comparable performance to the best results in the literature.