Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Does Rubix ML support Natural Language Processing (NLP)? #311

Open
programarivm opened this issue Dec 19, 2023 · 1 comment
Open

Does Rubix ML support Natural Language Processing (NLP)? #311

programarivm opened this issue Dec 19, 2023 · 1 comment

Comments

@programarivm
Copy link
Contributor

programarivm commented Dec 19, 2023 •
edited

Hi there,

At ChesslaBlab we're experimenting with different ways of rephrasing the robotic-like output of the PHP Chess Tutor into a user-friendly, human-like explanation. So far we've tried out a number of APIs such as ChatGPT, Google Bard as well as NLP Cloud.

Robotic-like text example:

White has a slightly better control of the center. The black pieces are significantly better connected. The white player is really pressuring more squares than its opponent. The white pieces are timidly approaching the other side's king. The pawn on b5 is unprotected. The pawn on f6 is doubled. The pawn on h7 is isolated. The pawns on a6, d6 and f6 are backward. d5, f5 and h3 are outpost squares. The knight on d5 is nicely placed on an outpost. Black has the bishop pair.

Human-like paraphrase:

White exhibits a minor edge when it comes to central control, while Black's pieces display a greater degree of connection. Additionally, White casts an intense pressure over a larger number of squares compared to the opponent. Yet, White's pieces are making somewhat tentative advances towards the enemy's King. The b5 pawn is left undefended. The f6 pawn is in double trouble, whereas the h7 pawn stands isolated. Pawns on a6, d6, and f6 are lagging behind. The squares at d5, f5, and h3 serve as outposts while the d5 knight stands poised at a strategic outpost. Finally, Black possesses the advantage of a bishop pair.

While the result obtained is okay I was wondering if Rubix ML could be used to train our own paraphrase generation model.

Any help will be very much appreciated!

Thank you,
@andrewdalpino
Copy link
Member

andrewdalpino commented Jan 22, 2024 •
edited

Hey @programarivm unfortunately Rubix ML does not make a good base library for a language generation project in my opinion. The reason is because language generation models, like the autoregressive type that predict the next word) tend to require a ton of computation especially at training but inference as well. Single-threaded CPU with additional PHP overhead would take forever to train. Additionally, we do not provide support for Transformers which is the current state-of-the-art neural network architecture at tackling these language generation problems.

I would take a look at fine-tuning a pretrained GPT with your chess-oriented corpus using PyTorch for your problem.

Where Rubix ML may be able to help you is in predicting intent from text. With that, you could respond with pregenerated text appropriately. This may or may not make your answers sound less robotic. It may even make it worse.

Hope all is well with you! Keep up the great work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@programarivm @andrewdalpino