In recent years, the issue of the elderly living alone is more and more serious in Taiwan; many older people with health issues need to be cared for a long time at home or home for aged. Accident prevention for the elderly is a very important issue. When people are busy with doing housework or taking a rest, we may put the elderly at risk. In this paper, we design a video streaming gateway system and a human tracking algorithm to real-time watch and track the status of the elderly who need to be cared. The gateway can decode H.264 video format transmitted from cameras and use background subtraction technique to do real time human tracking. The experimental results show that the gateway can support many cameras for real-time monitoring and human tracking.