This paper presents an ASP (application specific processor) with 512-bit SIMD (Single Instruction Multiple Data) and 192-bit VLIW (Very Long Instruction Word) architecture optimized for wireless baseband processing. It employs optimized architecture and address generation unit to accelerate the kernel algorithms. Based on the ASP, a multi-core baseband processor is developed which can work at 2x2 MIMO and 20 MHz physical bandwidth configuration for LTE inner receiver and meet requirements of Category 3 User Equipment (CAT3 UE). Furthermore, a silicon implementation of the baseband processor with 130nm CMOS technology is presented. Experimental results show that the baseband processor provides 100 GOPS computing ability at 117.6MHz.