Natural Language Processing (NLP) is the branch of AI that lets computers understand, interpret and generate human language, in text or speech. It combines linguistics with machine learning.
Major techniques
- Text pre-processing: tokenization, lower-casing, stop-word removal, stemming and lemmatization.
- Syntactic analysis: part-of-speech tagging, parsing (dependency / constituency trees).
- Semantic analysis: named-entity recognition (NER), word-sense disambiguation, coreference resolution.
- Text representation: bag-of-words, TF-IDF, word embeddings (Word2Vec, GloVe), contextual embeddings.
- Models: rule-based systems, statistical models (Naive Bayes, HMM, CRF), deep learning (RNN, LSTM) and Transformers (BERT, GPT) using self-attention.
Applications
- Machine translation (Google Translate), including Bangla ↔ English.
- Chatbots and virtual assistants; customer-support automation in banks.
- Sentiment analysis of reviews and social media.
- Spam and fraud detection, document classification.
- Text summarization, question answering, search engines.
- Speech recognition and text-to-speech.
— rhs