This is exactly what I'm doing:
% this loads 10 huge matrices, all of type double, same number of rows but different number of columns
for i = 1:10
load(['filename',num2str(i),'.mat']);
end
Then the computations:
% generate 7 random streams as messages
msg1 = (-1).^(rand(7,10000)>0.5);
% multiply each stream by a corresponding huge matrix 'x', I know that this is error but I provide it just to show what I'm trying to achieve.
for i = 1:n
msg2(i,:) = msg(i,:)*['x',num2str(i)];
end
As you see I have the same problem with my msg2 matrix since the rows will be of different lengths. I can preallocate it with the maximum length but then inside my code I need to extract the nonzero portion each time I'm using msg2 (more hassle).
Do I define everything in cells? I have not worked with cells so far nor with the structures. And, my concerns are the SPEED and the MEMORY. So any solution that gives the best of the two is the best solution for me.
I hope this makes everything more clear than it was before. And, thank you for your answers.
Elnaz
dpb <none@non.net> wrote in message <ltqa9r$jdj$1@dont-email.me>...
> On 08/29/2014 11:26 AM, Elnaz wrote:
> > Steven,
> > Just to be thorough, I use the matrices x1, ..., x10 thousands of times
> > in my code because I'm doing statistical averaging. And, each matrix is
> > about 800 MB in size. I do almost exact kind of computation with each of
> > them i.e. I multiply a different data vector with each matrix x1, x2,
> > ..., x10. That's why I did not want to load them each, use, and clear
> > them before loading the next one because loading each matrix takes 30
> > sec and that is too much for my code. So, I load them all in the
> > beginning of my code. Therefore, one part of the problem is the memory
> > shortage which I meanwhile can solve by running my code in a remote unix
> > server, and the second part of the problem is how to address each matrix
> > when I need it. Thank you very much for your answer above. Let me know
> > if you have any suggestion as to what is the best solution.
>
> Again, a _small_ example would undoubtedly be a useful adjunct to
> specific suggestions, but I see no reason why Steven's structure
> solution wouldn't work (nor, fundamentally, with the cell arrays,
> either, for that matter).
>
> And, depending on the size questions of the various x, doesn't seem to
> me that we've yet eliminated entirely the higher-dimensionality plain
> ol' array yet, either, for certain.
>
> --
>
% this loads 10 huge matrices, all of type double, same number of rows but different number of columns
for i = 1:10
load(['filename',num2str(i),'.mat']);
end
Then the computations:
% generate 7 random streams as messages
msg1 = (-1).^(rand(7,10000)>0.5);
% multiply each stream by a corresponding huge matrix 'x', I know that this is error but I provide it just to show what I'm trying to achieve.
for i = 1:n
msg2(i,:) = msg(i,:)*['x',num2str(i)];
end
As you see I have the same problem with my msg2 matrix since the rows will be of different lengths. I can preallocate it with the maximum length but then inside my code I need to extract the nonzero portion each time I'm using msg2 (more hassle).
Do I define everything in cells? I have not worked with cells so far nor with the structures. And, my concerns are the SPEED and the MEMORY. So any solution that gives the best of the two is the best solution for me.
I hope this makes everything more clear than it was before. And, thank you for your answers.
Elnaz
dpb <none@non.net> wrote in message <ltqa9r$jdj$1@dont-email.me>...
> On 08/29/2014 11:26 AM, Elnaz wrote:
> > Steven,
> > Just to be thorough, I use the matrices x1, ..., x10 thousands of times
> > in my code because I'm doing statistical averaging. And, each matrix is
> > about 800 MB in size. I do almost exact kind of computation with each of
> > them i.e. I multiply a different data vector with each matrix x1, x2,
> > ..., x10. That's why I did not want to load them each, use, and clear
> > them before loading the next one because loading each matrix takes 30
> > sec and that is too much for my code. So, I load them all in the
> > beginning of my code. Therefore, one part of the problem is the memory
> > shortage which I meanwhile can solve by running my code in a remote unix
> > server, and the second part of the problem is how to address each matrix
> > when I need it. Thank you very much for your answer above. Let me know
> > if you have any suggestion as to what is the best solution.
>
> Again, a _small_ example would undoubtedly be a useful adjunct to
> specific suggestions, but I see no reason why Steven's structure
> solution wouldn't work (nor, fundamentally, with the cell arrays,
> either, for that matter).
>
> And, depending on the size questions of the various x, doesn't seem to
> me that we've yet eliminated entirely the higher-dimensionality plain
> ol' array yet, either, for certain.
>
> --
>
