Fast Output-Sensitive Matrix Multiplication

We consider the problem of multiplying two

U

×

U

matrices

A

and

C

of elements from a field

$\mathcal{F}$

. We present a new randomized algorithm that can use the known fast square matrix multiplication algorithms to perform fewer arithmetic operations than the current state of the art for output matrices that are sparse.

In particular, let

ω

be the best known constant such that two dense

U

×

U

matrices can be multiplied with

$\mathcal{O} \left( U^\omega \right)$

arithmetic operations. Further denote by

N

the number of nonzero entries in the input matrices while

Z

is the number of nonzero entries of matrix product

AC

. We present a new Monte Carlo algorithm that uses

$\tilde{\mathcal{O}} \left( U^2 \left(\frac{Z}{U}\right)^{\omega-2} + N \right)$

arithmetic operations and outputs the nonzero entries of

AC

with high probability. For dense input, i.e.,

N

 = 

U

2

, if

Z

is asymptotically larger than

U

, this improves over state of the art methods, and it is always at most

$\mathcal{O} \left( U^\omega \right)$

. For general input density we improve upon state of the art when

N

is asymptotically larger than

U

4 − 

ω

Z

ω

 − 5/2

.

The algorithm is based on dividing the input into ”balanced” subproblems which are then compressed and computed. The new subroutine that computes a matrix product with balanced rows and columns in its output uses time

$\tilde{\mathcal{O}} \left( U Z^{(\omega -1)/2} + N\right)$

which is better than the current state of the art for balanced matrices when

N

is asymptotically larger than

U

Z

ω

/2 − 1

, which always holds when

N

 = 

U

2

.

In the I/O model — where

M

is the memory size and

B

is the block size — our algorithm is the first nontrivial result that exploits cancellations and sparsity of the output. The I/O complexity of our algorithm is

$\tilde{\mathcal{O}} \left( U^2 (Z/U)^{\omega-2}/(M^{\omega/2 -1} B) + Z/B + N/B \right)$

, which is asymptotically faster than the state of the art unless

M

is large.