In the United States, building sector is responsible for around 40% of total energy consumption and contributes about 40% of carbon emissions since 2012. Within the past several years, various optimization models and control strategies have been studied to improve buildings’ energy efficiency and reduce operational expenses under the constraints of satisfying occupants’ comfort requirements. However, the majority of these studies consider building electricity demand and thermal load being satisfied by unidirectional electricity flow from the power grid or on-site renewable energy generation to electrical and thermal home appliances. Opportunities for leveraging low-grade heat for electricity have largely been overlooked due to impracticality at small scale. In 2016, a modular pumped hydro storage technology was invented in Oak Ridge National Laboratory, named Ground Level Integrated Diverse Energy Storage (GLIDES). In GLIDES, employing high-efficiency hydraulic machinery instead of gas compressor/turbine, liquid is pumped to compress gas inside high-pressure vessel creating head on ground level. This unique design eliminates the geographical limitation associated with the existing state-of-the-art energy storage technologies. It is easy to be scaled for building level, community level, and grid level applications. By using this novel hydro-pneumatic storage technology, opportunities for leveraging low-grade heat in building can be economical. In this research, the potential of utilizing low-grade thermal energy to augment electricity generation of GLIDES is investigated. Since GLIDES relies on gas expansion in the discharge process and the gas temperature drops during this nonisothermal process, available thermal energy, e.g., from thermal storage, combined cooling, heat and power system (CCHP), can be utilized by GLIDES to counter the cooling effect of the expansion process and elevate the gas temperature and pressure and boost the roundtrip efficiency. Several groups of comparison experiments have been conducted, and the experimental results show that a maximum 12.9% cost saving could be achieved with unlimited heat source for GLIDES, and a moderate 3.8% cost improvement can be expected when operated coordinately with CCHP and thermal energy storage in a smart building.