On this page
← All work

Semi-supervised Regulatory NLP

Semi-supervised text classification for regulatory documents using IndoBERT with contrastive learning and pseudo-labeling, where labeled data is scarce.

Research Project · Institut Teknologi Sepuluh Nopember2025
  • NLP
  • Semi-supervised Learning
  • Contrastive Learning
  • IndoBERT

An exploration of learning from unlabeled Indonesian regulatory text when gold labels are expensive.

Problem

Regulatory documents in Indonesian need classification, but annotation is slow and expert-dependent. A practical classifier must get by with very few labels.

Research Question

Can semi-supervised learning — pseudo-labeling plus contrastive pretraining on unlabeled text — match supervised performance with a fraction of the labeled data?

System

Unlabeled regulatory text ──► contrastive pretraining (IndoBERT)
                                      │
Small labeled set ──► fine-tune ──► pseudo-label the rest
                                      │
                              retrain / iterate
                                      │
                                      ▼
                              Evaluation (held-out)
  • IndoBERT as the backbone encoder.
  • Contrastive learning over unlabeled documents to learn useful representations.
  • Pseudo-labeling loop to expand the effective training set.

My Contribution

  • Preprocessing pipeline for Indonesian regulatory text.
  • Contrastive training setup and data augmentation for positive pairs.
  • Pseudo-labeling loop with confidence thresholds and iteration.
  • Evaluation across label-set sizes to show the data-efficiency curve.

Experiments

Trained with 1%, 5%, 10%, and 25% of the labeled data, comparing supervised vs semi-supervised regimes.

Results

Semi-supervised training recovered most of the supervised performance at low label budgets; contrastive pretraining helped most in the extreme low-label setting.

Results are placeholder text — replace with actual numbers.

What I Learned

  • Label efficiency is a systems decision: augmentation, confidence thresholding, and iteration matter.
  • Contrastive objectives are a strong fit when unlabeled data is abundant.
  • Pseudo-labeling is sensitive to threshold calibration.