Institute of Smart Systems and Artificial Intelligence at Nazarbayev University was pleased to welcome Alibaba Group, who conducted a workshop for ISSAI Data Scientists on March 27, sharing valuable insights into practical approaches for developing and adapting large language models for low-resource languages.

The workshop highlighted several important technical directions for building more efficient and culturally grounded AI systems:

• Data Processing & Synthetic Expansion
  The team described an approach that begins with Wikipedia as seed data for the target language and then expands it synthetically. Importantly, the focus is not simply on generating more data, but on generating the right data.
• Model Merging
  Another topic was model merging, where two models are trained separately: one on the low-resource language and another as the original base model, and then merged. This approach helps preserve newly acquired language capabilities while avoiding catastrophic forgetting of the base model’s broader reasoning abilities.
• Pin-Point Learning
  A particularly interesting area of focus was pin-point learning. Instead of fine-tuning an entire model, this method uses mechanistic interpretability to identify the specific parameters responsible for language expression. Most parameters remain frozen, while only the relevant ones are updated. Alibaba Group representatives also shared findings suggesting that the final layers of the model play an important role in translating internal representations into target-language output, showing how interpretability can be used not only to understand models, but also to make training more efficient and targeted.

Workshops like this play an important role in strengthening international knowledge exchange and advancing practical AI research for multilingual and low-resource settings.

We sincerely thank Alibaba Group for this productive and insightful workshop, and for sharing their expertise with the ISSAI team.