The brain can perform massively parallel information processing while consuming only ~1- 100 fJ per synaptic event. Two-terminal memristors based on filament forming metal oxides (FFMOs) or phase change memory (PCM) materials have recently been demonstrated to function as non-volatile memory that can emulate neuronal and synaptic functions. Despite recent progress in the fabrication of device arrays however, to date no architecture has been shown to operate with the projected energy efficiency while maintaining high accuracy. A major impediment still exists at the device level, specifically, a resistive memory device has not yet been demonstrated with adequate electrical characteristics to fully realize the efficiency and performance gains of a neural architecture. I will describe a novel electrochemical neuromorphic device (ENODe) that switches at record-low energy (<0.1 fJ projected, <10 pJ measured) and voltage (< 1mV, measured), displays >500 distinct, non-volatile conductance states within a ~1 V operating range, and achieves record classification accuracy when implemented in neural network simulations. We recently showed that combined with a Si access device we are able to achieve over 106 switching events with very little degradation. I will discuss the working mechanism and paths towards further improvement in performance and stability.