Summary: Community logistics (CL) is a recently proposed delivery strategy designed to deal with e-commerce last-mile delivery scheduling by dynamically assigning vehicles to designated delivery regions partitioned into “communities”. Since optimizing vehicle routes is not mandatory in the CL spectrum, the delivery solution format and optimization process can be greatly simplified. Nevertheless, abandoning vehicle routes means vehicle arrival time at each customer specified delivery destination is unknown, resulting in the inability of handling time window constraints of e-commerce orders. To expand the application scope of CL, this study introduces community time window, an aggregation of identical or adjacent order time windows. Once the community time window for a delivery community is satisfied, all orders in this community can be received within designated time windows without determining vehicle routes. With this new concept, the application range of the CL is extended to e-commerce last mile delivery contexts where order time window constraints are considered. A dynamic community partitioning problem with the time window is presented based on the Markov decision process (MDP). An efficient heuristic solution framework based on policy function approximation is proposed to solve the MDP model. Numerical results show that the CL is very effective in dealing with the time window constraints of e-commerce orders.