{"id":286,"date":"2023-12-28T00:06:32","date_gmt":"2023-12-27T20:36:32","guid":{"rendered":"https:\/\/haghiri75.com\/en\/?p=286"},"modified":"2023-12-28T00:06:32","modified_gmt":"2023-12-27T20:36:32","slug":"maral-is-here-7-billion-parameters-bilingual-model-with-support-of-persian","status":"publish","type":"post","link":"https:\/\/haghiri75.com\/en\/maral-is-here-7-billion-parameters-bilingual-model-with-support-of-persian\/","title":{"rendered":"Maral is here, 7 billion parameters bilingual model with support of Persian!"},"content":{"rendered":"

If you read my previous post, you know how much I like open source AI material, and I even jokingly titled my BLOOM post I was too cheap to pay for GitHub’s copilot<\/a>! So making an open source model was always one of my goals of life. Also, in my Persian blog, I pointed out that the dominance of English language in current LLM scene is a little bit concerning (read it here<\/a>).<\/p>\n

Now as of today, I am pleased to announce that Maral is here. The 7 billion parameters bilingual model which can respond to Persian and English prompts, and can produce GPT-3.5 level of answers based on the dataset we fed to it!<\/p>\n

\"\"<\/p>\n

Maral 7B alpha 1 and its advantages<\/h1>\n

Since the release of GPT2 and BERT models, there were efforts for making a Persian text generation<\/em> model in our community. But to be honest, most of them left untouched in middle of the road.<\/p>\n

In last years AI revolution <\/em>however, people saw potential in the realm of generative AI and started working on models. From RAGs on existing models to fine-tuning basic models which could somehow understand Perso-Arabic alphabet.<\/p>\n

But with the release of Mistral<\/a> model, everything has changed. I personally never thought a 7 billion parameters model can understand multiple languages this well. So I put more information on the next section of the article on why Mistral became my number one choice as the base model!<\/p>\n

However, the biggest problem was still there and it was the dataset.<\/em> Finding a good enough dataset is always a bottleneck. But we’ve been lucky enough that one of Iranian developers, has translated Alpaca Dataset <\/em>to our beloved Persian language (and it’s accessible here<\/a>).<\/p>\n

When you’re in possession of needed ingredients for your potion, I guess it’s time to light up the caldron and start making the potion!<\/p>\n

Why Mistral?<\/h2>\n

As a developer and an enthusiastic person, I always try new models and tools specially when it comes to text. Mistral was the new kid in the corner and I personally witnessed a lot of positive reviews about it. So I tried these:<\/p>\n