The Internet of Things (IoT) is a massively extensive environment that can manage many diverse applications. Security is critical due to potential malicious threats and the diversity of the connectivity. Devices can protect themselves and detect threats with the Intrusion Detection System (IDS). IDS typically uses one of two approaches: anomaly-based or signature-based. This paper proposes a model (known as “AS-IDS”) that combines these two approaches to detect known and unknown attacks in IoT networks. The proposed model has three phases: traffic filtering, preprocessing and the hybrid IDS. In the first phase, the arrival traffic is filtered at the IoT gateway by matching packet features, after which the preprocessing phase applies a Target Encoder, Z-score and Discrete Hessian Eigenmap (DHE) to encode, normalize and eliminate redundancy, respectively. In the final phase, the hybrid IDS integrates signatures and anomalies. The signature-based IDS subsystem investigates packets with Lightweight Neural Network (LightNet), which uses Human Mental Search (HMS) for traffic clustering in the hidden layer and Boyer Moore is used to search for a particular signature in the output layer that is accelerated by using the Generalized Suffix Tree (GST) algorithm and by matching the signatures it classifies the attacks as intruder, normal or unknown. The anomaly-based IDS subsystem employs Deep Q-learning to identify unknown attacks, and uses Signal to Noise Ratio (SNR) and bandwidth to classify the attacks into five classes: Denial of Service (DoS), Probe, User-to-Root (U2R), Remote-to-Local (R2L), and normal traffic. Detected packets are then generated with new signatures, using the Position Aware Distribution Signature (PADS) algorithm. The proposed AS-IDS is implemented in real-time traffic with the NSL-KDD dataset, and the results are evaluated in terms of Detection Rate (DR), False Alarm Rate (FAR), Specificity, F-measure and computation time.