Vector spaces are one of the most common mathematical abstractions. In classical physics, vectors can describe linear motion in space. In operations research, vectors can describe linear constraints to optimization problems. In statistics and machine learning, vectors can describe features in datasets.

In the aforementioned use cases, it is normally sufficient to represent vectors as some array of real values. The rigidity of arrays is the major flaw of this approach: how do you extend arrays to support other functionality such as user-defined annotations? Furthermore, how can we maintain type safety so that the compiler can infer common features of vector spaces? We introduce a simple vector interface to solve this issue.

interfaceVector<EextendsVector<E>>{Eadd(EotherVector);Emultiply(doublescalar);defaultEadditiveInverse(){returnmultiply(-1.0);}defaultEsubtract(EotherVector){returnadd(otherVector.additiveInverse()));}}

Before diving into the creation of this interface, we begin with an example vector space to motivate the problem.

Polynomials Example

Consider the following problem: we want a library to represent single variable polynomials of the form:

The implementation is straightforward:

// Represents f(x) = a_n * x^n + a_{n-1} * x^{n-1} + ... + a_1 * x + a_0classPolynomialimplementsFunction<Double,Double>{// Represents a_0, a_1, ..., a_nprivatefinaldouble[]coefficients;publicPolynomial(double[]coefficients){this.coefficients=coefficients;}@OverridepublicDoubleapply(Doublex){doubleresult=coefficients[0];for(inti=1;i<coefficients.length;i++){result+=coefficients[i]*Math.pow(x,i)}returnresult;}}

We now simulate two sets of modifications to the requirements of this problem to show the development and use of the Vector interface:

1. Add vector addition and scalar multiplication
2. Add vector subtraction and additive inversion.

Polynomial Operations

Our first change to the requirements is the addition of two new operations: adding polynomials by their coefficients and scaling a polynomial's coefficients by a scalar value. With these operations, we can consider polynomials as vectors in a vector space. To develop some intuition for these operations, we sketch some examples.

Polynomial Addition

Adding two polynomials, $a$ and $b$, results in a new polynomial where the coefficients have been summed together component-wise.

Polynomial Scalar Multiplication

Multiplying a polynomial, $a$, by a scalar, $\lambda$ means that we scale each coefficient of the polynomial, $a_i$, by the scalar.

Implementation of Polynomial Operations

The implementations of vector addition and scalar multiplication for this problem is trivial. The operations are defined below:

// vector additionpublicPolynomialadd(Polynomialother){finaldouble[]newCoefficients=coefficients.clone();for(inti=0;i<coefficients.length;i++){newCoefficients[i]=newCoefficients[i]*other.coeffcients[i];}returnnewCoefficients;}// scalar multiplicationpublicPolynomialmultiply(doublescalar){finaldouble[]newCoefficients=coefficients.clone();for(inti=0;i<coefficients.length;i++){newCoefficients[i]=newCoefficients[i]*scalar;}returnnewCoefficients;}

Polynomial Subtraction and Additive Inversion

The second modification to the requirements introduce vector subtraction and additive inversion. Vector subtraction means that we can subtract the coefficients of two polynomials. Additive inversion means that we negate the coefficients of a polynomial. It is trivial to show that we can define vector subtraction and additive inversion in terms of vector addition and scalar multiplication.

// vector subtractionpublicPolynomialsubtract(Polynomialother){returnadd(other.multiply(-1.0));}// additive inversionpublicPolynomialadditiveInverse(){returnmultiply(-1.0);}

Lessons Learned

During the construction of our polynomial vector space, we realize that we must implement two operations: vector addition and scalar multiplication. Furthermore, we may infer the implementation of vector subtraction and additive inversion by the implementation of vector addition and scalar multiplication.

This problem appears to generalize well into a library where we can use any object that satisfies the base set of vector space properties in a variety of vector-based algorithms such as models of motion, constrained optimization, and clustering.

Modeling Vector Spaces

Before we introduce any code, we begin by analyzing vector spaces from its mathematical definition as an algebraic structure. Frequently, mathematical abstractions provide a good starting point for modeling by defining the objects needed to model as well as the operations between the model.

Abstract Algebra is the field of mathematics that is concerned with the algebraic structures. Algebraic structures are sets with operations defined on the elements of the set and other optional sets.

Mathematical Vector Spaces

Formally, a vector space contains two types of objects: vectors, $v \in \mathbf{V}$, and scalars, $f \in \mathbf{F}$. Vector spaces have two operations: vector addition and scalar multiplication. In function notation, these two operations can be described below:

Function notation describes operations by the types of its arguments and results. The types of arguments are to the left of the $\mapsto$ symbol and $\times$ symbol delimits the types of arguments. The types of results are to the right of the $\mapsto$ symbol and the $\times$ symbol also delimits the types of arguments.

The mathematical definition of vector spaces provides sufficient details about the types we need to model in addition to the operations supported by those types.

Vector Space Interface

For modeling scalars, it is sufficient to use double primitives. However, The implementation of vectors may vary depending on the problem at hand. Trivially, we can represent a vector as an array of doubles, double[]. Unfortunately, this representation is rigid and difficult to extend. We propose an interface instead.

To construct this interface, we begin by modeling the operations. Vector addition, $+$, consists of adding two vector objects to produce a new vector object. Scalar multiplication, $\cdot$, consists of multiplying a vector object with a scalar to produce a new vector object. We show a rough sketch of an interface below:

interfaceVector{Vectoradd(VectorotherVector)Vectormultiply(doublescalar)}

This interface, however, is unsatisfactory because vector implementations are forced to handle interoperability with other vector types. In mathematics, however, it is not the case that a vector is required to interact with vectors from other spaces. To mitigate this problem, we use Java generics with the Curiously Recursive Template Pattern (CRTP) to parameterize the type by its subtypes.

interfaceVector<EextendsVector<E>>{Eadd(EotherVector)Emultiply(doublescalar)}

This interface only requires vectors from the same space (subtype) to interact with other vectors in its space.

Default Implementations

Suppose we would like this interface to have default implementations for operations that can be defined in terms of other operations.

// additive inversion publicPolynomialadditiveInverse(){returnmultiply(-1.0);}// vector subtractionpublicPolynomialsubtract(Polynomialother){returnadd(other.additiveInverse());}

It is important that we highlight the CRTP pattern here. Without knowing that E is a subtype of Vector<E>, we would not be able to provide default implementations for operations that depend on other abstracted operations. This problem occurs in Java because of type erasure.

Type erasure is a feature of Java where generic types are erased at runtime. This means that unbounded objects of E can only be assumed to have the same interface as Object.

With these interface constraints and default operations, we have completed the interface.

Polynomials Example Revisited

We revisit the implementation of the Polynomial class so that it implements Vector<E> interface.

// Represents f(x) = a_n * x^n + a_{n-1} * x^{n-1} + ... + a_1 * x + a_0classPolynomialimplementsVector<Polynomial>,Function<Double,Double>{// Represents a_0, a_1, ..., a_nprivatefinaldouble[]coefficients;publicPolynomial(double[]coefficients){this.coefficients=coefficients;}@OverridepublicDoubleapply(Doublex){doubleresult=coefficients[0];for(inti=1;i<coefficients.length;i++){result+=coefficients[i]*Math.pow(x,i)}returnresult;}@Override// vector additionpublicPolynomialadd(Polynomialother){finaldouble[]newCoefficients=coefficients.clone();for(inti=0;i<coefficients.length;i++){newCoefficients[i]=newCoefficients[i]*other.coeffcients[i];}returnnewCoefficients;}@Override// scalar multiplicationpublicPolynomialmultiply(doublescalar){finaldouble[]newCoefficients=coefficients.clone();for(inti=0;i<coefficients.length;i++){newCoefficients[i]=newCoefficients[i]*scalar;}returnnewCoefficients;}}

This implementation no longer requires the vector subtraction nor the additive inverse implementations because the interfaces provides them.

Interface for Algorithms

In addition to modeling vector-based types, this interface provides a foundation for generic vector-based algorithms including optimization and matrix factorization. It is easy to conceive a gradient descent optimization algorithm based on this vector interface.

classGradientDescent<EextendsVector<E>>{privatedoublemaxIterations;privatedoublelearningRate;// Getters and setters for propertiespublicEoptimize(EinitialState,Function<E,E>gradient){Estate=initialState;for(intt=0;t<maxIterations;t++){state=state.subtract(gradient.apply(state).multiply(learningRate));}returnstate;}}

The GradientDescent implementation can now perform optimization on any user-defined type that implements Vector<E> and some notion of a gradient.

Conclusion

The Vector<E> interface provides an extensible and type safe representation of vector spaces in Java 8 as we have seen with the polynomials example. This theory should also extend to other vector formulations of user-defined types.

Future work for this interface include the introduction of algorithms backed by this interface such as gradient descent. I will continue working on this interface as well as libraries for it in my Github repository vec. The vision for this project is to enable software engineers to formulate and solve problems in terms of vectors rather than arrays-of-doubles.