The use of classification techniques for machine health monitoring and fault diagnosis has been popular in recent years. System response in the form of time series data can be used to identify the type of defect and severity of defect. However, a central issue with time series classification is that of identifying appropriate features for classification. In this paper, we explore a new feature set based on delay differential equations (DDEs). DDEs have been used recently for extracting features for classification but have never been used to classify system responses. The Duffing oscillator, Van der Pol–Duffing (VDP-D) oscillator, Lu oscillator, and Chen oscillator are used as examples for dynamic systems, and the responses are classified into self-similar groups. Responses with the same period should belong to the same group. Misclassification rate is used as an indicator of the efficacy of the feature set. The proposed feature set is compared to a statistical feature set, a power spectral coefficient feature set, and a wavelet coefficient feature set. In the work described in this paper, a density-estimation algorithm called DBSCAN is used as the classification algorithm. The proposed DDE-based feature set is found to be significantly better than the other feature sets for classifying responses generated by the Duffing, Lu, and Chen systems. The wavelet and power spectral coefficient data sets are not found to be significantly better than the statistical feature set for these systems. None of the feature sets tested is discerning enough on the VDP-D system.