After British army arrival at Possession Point (now Hollywood Park next to Possession Street) in 1841, the 800 or so volunteer soldiers from India and Bangladesh stationed in West Point (hillside of Water Street). Thereafter, the area west of Hollywood Park was called Sai Ying Pun 西營盤, meaning 'Western Garrison'. Flourishing trades required construction of piers, docks, warehouses, and properties to accommodate merchants for rice, dry seafoods, and herbal medicines, and the workers who supported these activities. Even today, one can still find remnants of the early colonial past in the district. World War II brought mass destruction, giving rise to reconstruction of many buildings to four to six storey tenement buildings with shops on ground floor. Rapid population growth and technological advancement necessitate these post war buildings to be redeveloped into highrises.

There are many prestigious colleges and schools can be found in this district, namely King's College, St. Clare Primary, St. Stephen Girls College, St. Paul College, etc.

Property values were suppressed as MTR somehow missed Sai Wan when the railway opened in 1979. Thanks to recent MTR access, the long forgotten sleepy Sai Wan is coming back in a vengeance. As old buildings are redeveloped, new home buyers and tenants are moving in, which attract new and upscale businesses to serve this new and more affluent customer base. The district is now flourishing as many new businesses are setting up, including chain stores, specialty stores, and trendy restaurants and pubs.