Rust port?

[asdofindia]

Is a Rust port of varnam a bad idea? If someone's porting varnam into a different language, what should they do differently?

[joicemjoseph]

We(Subin, Kailash and me) had a plan to port Varnam to Go. Also had a conversation with Navaneeth on the architecture as well.
AFAIK, core of this project is a lexer(I guess) written in Ruby. It leverages advantages of that language. porting the rule engine will be the a bottleneck I guess.
It will be great to port to modern languages, either go or rust.

[subins2000]

If we're going to do a rewrite, some improvements I suggest :

• No need of a pattern => word DB (learnings file). Instead, just need a word dictionary.
• Add more patterns to VST (Varnam Symbol Table). Prioritized letters n => ന, ണ. Capitalized N will always give ണ. So pani will give suggestions in priority : പനി, പണി. Currently if only the learnings DB has pani assigned to both words will give the different outputs.
• When an input say pani is given to varnam, it should tokenize to പനി and പണി using just VST, and then look up the word dictionary to find words starting with പനി and പണി and give additional suggestions.
• For english words like "Cricket", the tokenization will give bad results, in such cases we can maybe use a pattern => word DB like the current learnings DB.

[subins2000]

Also, can dynamic libraries be made in Go & Rust ? Can it then be compiled in Android to be used in Indic Keyboard ? A straight replacement ? Is there a JNI equivalent for Go/Rust libraries in Java ? If not, things will be more difficult.

[asdofindia]

There's https://github.com/mozilla/rust-android-gradle for rust in android

[subins2000]

Varnam desktop app, varnamd API server is all written in Go. If we're going to do a rewrite, golang will be the better choice for compatibility reasons and maintainability. I saw that there is JNI compatibility with go too.

I hope it will be a straight, simple replacement or others that depend on libvarnam breaks! (Desktop, Indic Keyboard, API) 😅

[asdofindia]

@joicemjoseph , you're right about the lexer. https://asd.learnlearn.in/varnam-1/