The Spatial Augmented Reality (SAR) system is an interactive platform that allows any virtual contents to be projected onto any physical structure (object). However, most of the SAR platforms use pre-rendered models in a static setting that does not allow the physical structure to be modified dynamically. This has limited interactions between the users and the system because users are unable to make rapid changes to the physical structure to express their ideas. This limitation can be a huge challenge when it comes to application such as city planning, where rapid real-time prototyping is able to provide a better visualization to the impacts that the changes could bring to the city environment. Therefore, this research project aims to design and develop a tracker-based SAR system to resolve the aforementioned limitation. The main contributions of this research project include, (i) a SAR system that supports real-time physical structure reconstruction and projection mapping, and (ii) a SAR platform constructed using Lego blocks and easily accessible hardware and software. The hardware involves the design of the physical set-up to support a real time reconstruction, and dynamic projection mapping. The software involves real time object detection, tracking, and projection mapping. Real time object detection is carried out using colour tracking, and recording Lego positions, while dynamic projection mapping is done through marker tracking and coordinate mapping. Based on preliminary evaluations conducted in the laboratory, the experimental results shown that the proposed SAR system is able to (i) successfully project virtual content onto physical structure built using Lego blocks in real-time, and (ii) detect changes made to the physical structure.