Large Language Models (LLMs) represent a major advance in the field of deep learning. Their ability to understand long-term dependencies between words in a sentence has completely revolutionized natural language processing. Based on the architecture of transformers, LLMs are trained to solve common linguistic problems, such as text classification, question answering, text generation, document summarization, etc. LLMs can be pre-trained and then fine-tuned to perform tasks in specific domains. In the field of cybersecurity, we have proposed in our previous work, a LLM-based approach for web applications vulnerabilities detection. In this paper, we implement our approach, which uses adaptation techniques such as fine-tuning to detect and classify common web attacks like XSS, Directory Traversal, SQL injection, Command Injection and benign payloads recognition. LLMs used for fine-tuning are : BERT (encoder model), Llama (decoder model), Flan-T5 (encoder-decoder model). This is one of the first comparative evaluations of BERT, Llama, and Flan-T5 fine-tuned on real-world web attack payloads. We used several fine-tuning techniques including Quantization, Low-Rank Adaptation, Supervised Fine-Tuning or freezing the weights of the LLMs layers and gradually adding fully connected layers. For each of the LLMs, we obtained an accuracy rate of over 98% after fine-tuning. Since we were working with unbalanced dataset, we also evaluated models using metrics such as Balanced Accuracy, Geometric Mean, Matthews Correlation Coefficient, Fowlkes-Mallows Index. The fine-tuned BERT and Flan-T5 models proved to be more robust to unbalanced data. Evaluation with the confusion matrix shows that fine-tuned models correctly classify web application attacks. This confirms that fine-tuning basic models is effective for specific cybersecurity tasks.

Adaptation models, Accuracy, Large language models, Text categorization, Bidirectional control, Transformers, Encoding, Service-oriented architecture, Computer security, Payloads, Fine-tuning;Large Language Model, Web application attacks, Detection