Quantcast
Channel: Math.NET Numerics
Viewing all articles
Browse latest Browse all 971

New Post: How to load data into matrix

$
0
0
Christoph,

re: your comment:

"D) Consider dense matrices

If I change the above example to use a dense matrix instead (after all such a 100M cell matrix still fits into memory easily):

let dense = new DenseMatrix(10500,10500)
for row in 0..999 do
    for col in 0..999 do
        dense.At(10*row, 10*col, float32 (Math.Atan2(float row, float col)))
        dense.At(2+10*row, 3+10*col, float32 (row+col))

then the example runs in less than a second. However, any operation on that matrix would obviously then be very inefficient, but it depends what you actually want to do with it."


Our next operation on this bad boy is:

D = (SparseMatrix)(I - A).Inverse();

where I is the identity matrix also (10500, 10500). If you have suggestions on improving this code, we welcome your comment.


When we went from working with 800x800 matrices to the dim 10500, we hit out of memory problems just initializing the I matrix. So we pulled in the latest Math.Net libraries and switched most of the matrices to Sparse except for those over 50% non-zero ... but maybe we should try setting them back to Matrix or as DenseMatrix.

When you say "(after all such a 100M cell matrix still fits into memory easily)" - do you have guideline for selecting use of sparse vs. dense? I (simpleton) just computed number of non-zero datapoints as a percentage to determine whether the thing was sparse or not. Do you have a smarter guideline we should be using?

Thanks for all your help!

Yvonne




On Tue, Nov 27, 2012 at 8:20 AM, Yvonne Burgess <yvonne@climateearth.com> wrote:
Thanks a bunch, Christoph.

We'll look at this and let you know what we come up with.

Yvonne


On Tue, Nov 27, 2012 at 6:02 AM, cdrnet <notifications@codeplex.com> wrote:

From: cdrnet

Quick note: we currently have a [url:pull request|https://github.com/mathnet/mathnet-numerics/pull/58] open for an alternative sparse storage scheme (based on Knuth) that might be much more efficient in this scenario, since it seems to avoid almost all copying in this case. Might be worth to give it a trial.

Thanks,
Christoph

Read the full discussion online.

To add a post to this discussion, reply to this email (mathnetnumerics@discussions.codeplex.com)

To start a new discussion for this project, email mathnetnumerics@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com




--
--
Yvonne Burgess
Chief Systems Architect
Climate Earth, Inc.
415.391.2725 (office)
415.816.2674 (mobile)
www.climateearth.com




--
--
Yvonne Burgess
Chief Systems Architect
Climate Earth, Inc.
415.391.2725 (office)
415.816.2674 (mobile)
www.climateearth.com


Viewing all articles
Browse latest Browse all 971

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>