spago - Optimize machine learning in Go with Spago, supporting neural architectures and NLP

Introduction to Spago

Spago is an innovative machine learning library specifically crafted for the Go programming language. Its primary focus is on natural language processing, making it an excellent tool for developers interested in building applications that comprehend or generate human language. Remarkably, Spago operates independently, employing its unique computational graph system to facilitate both training and inference processes. This approach ensures simplicity and clarity from the onset to the conclusion of any project.

Key Features

Spago boasts a variety of features that make it a robust choice for machine learning endeavors:

Automatic Differentiation: Utilizing dynamic define-by-run execution, Spago efficiently performs automatic differentiation.
Diverse Neural Layers: The library supports various neural architectures, including feed-forward layers (like Linear, Highway, and Convolution), recurrent layers (such as LSTM, GRU, BiLSTM), and attention layers (like Self-Attention and Multi-Head Attention).
Optimization Techniques: Users can leverage a range of gradient descent optimizers, including Adam, RAdam, RMS-Prop, AdaGrad, and SGD.
Serialization: Spago offers compatibility with Gob, facilitating the serialization of neural models.

Practical Usage

Before diving into Spago, make sure you meet the necessary requirements, the most critical being Go version 1.21. Users can download the library by executing the following command:

go get -u github.com/nlpodyssey/spago

Getting Started

Begin your journey with Spago by examining the built-in neural models, such as the Long Short-Term Memory (LSTM) model, which is fundamental in the realm of recurrent neural networks.

Example: Calculating the Sum of Two Variables

Below is a simple example illustrating how to compute the sum of two variables using Spago:

package main

import (
	"fmt"
	"log"

	"github.com/nlpodyssey/spago/ag"
	"github.com/nlpodyssey/spago/mat"
)

func main() {
	type T = float32

	a := mat.Scalar(T(2.0), mat.WithGrad(true))
	b := mat.Scalar(T(5.0), mat.WithGrad(true))
	c := ag.Add(a, b)

	fmt.Printf("c = %v (float%d)\n", c.Value(), c.Value().Item().BitSize())

	c.AccGrad(mat.Scalar(T(0.5)))

	if err := ag.Backward(c); err != nil {
		log.Fatalf("error during Backward(): %v", err)
	}

	fmt.Printf("ga = %v\n", a.Grad())
	fmt.Printf("gb = %v\n", b.Grad())
}

Output:

c = [7] (float32)
ga = [0.5]
gb = [0.5]

Example: Implementing the Perceptron Formula

Here's a concise implementation of the perceptron formula, a basic yet powerful model used in machine learning:

package main

import (
	"fmt"
	
	. "github.com/nlpodyssey/spago/ag"
	"github.com/nlpodyssey/spago/mat"
)

func main() {
	x := mat.Scalar(-0.8)
	w := mat.Scalar(0.4)
	b := mat.Scalar(-0.2)

	y := Sigmoid(Add(Mul(w, x), b))

	fmt.Printf("y = %0.3f\n", y.Value().Item())
}

Contributing to Spago

Spago thrives on community involvement. If you identify areas for improvement or wish to contribute new features, you're encouraged to engage through issues and pull requests. For guidelines on contributing, consult the Contributing Guidelines.

Contact and Community Engagement

Building a community around Spago is a priority, and user contributions play a vital role. Those who prefer private communication can reach out to Matteo Grella via email. Whether through public issues or private messages, your input is invaluable to the project's growth and success.

In summary, Spago represents a leap forward in making machine learning accessible and effective in the Go programming landscape, equipped with tools and features that cater to a wide array of NLP applications.