In ancient Chinese manuscript states that the mosque was built by Sa'd ibn Abi Waqqas who is also the uncle of the Prophet Muhammad and is the first Muslim missionaries in China in 630's. Although these data are hard to find in other manuscripts, was not even known that Sa'd Ibn Abi Waqqas never actually visited the country of China, but the spread of Islam in the country's bamboo curtain begins in the 7th century, the Tang Dynasty, or even older, the Song dynasty.
The mosque is already 4 times in the renovation in accordance with the explanations contained inscriptions in the courtyard of the Yuan Dynasty, Ming Dynasty, Qing Dynasty and the JAR in 1935. Overall, the mosque was rebuilt in the days 1350M rule under the Yuan dynasty emperor Zhizheng (1341M-1368M) and rebuild again in the 1695M in the reign of Qing Dynasty Emperor Kangzi after destroyed by fire.