The consistency of the features machined by the recently developed nanoscale electromachining process (nano-EM) on the scanning tunneling microscope (STM) platform depends on the quality of the tools used. A simple, fast and reliable method has been developed in situ to evaluate nano-EM tool quality using current-displacement (I-Z) spectroscopy curves. Using this quality criterion and High Resolution Transmission Electron Microscopy (HR-TEM), the "health" of nano-EM tools was analyzed before and after machining for wear studies and their subsequent usability. From a manufacturing scalability perspective, it is imperative that this novel machining process demonstrates good repeatability. A repeatability study of the performance of nano-EM was carried out for the first time using different nano-tools and forms the basis of this research. It was found that the spread of the features in X, Y and Z dimensions was <20%, which implies that nano-EM has the potential to be a commercial nanomanufacturing process.