Discussion:
Array indexing: what is IDL doing?
(too old to reply)
Conor
2008-06-06 16:50:33 UTC
Permalink
So I want to pull out a small subsection from within a larger idl
array. I want to do something like this:

bigarr = findgen(500,100000)
bigx = [indgen(50)+100,indgen(25)+200]
bigy = indgen(10000)+50000

res = bigarr[bigx,bigy]

However IDL doesn't like that and says:

% All array subscripts must be same size. Var = BIGARR
% Execution halted at: $MAIN$

So I have to come up with another way to do it. My two thoughts were
the most straight forward way:

tarr = bigarr[bigx,*]
res = tarr[*,bigy]

and my other thought was to pre-build the indexes I want to extract,
and so I came up with this relatively simple command:

cx = n_elements(wx)
cy = n_elements(wy)
nx = n_elements(bigarr[*,0])
ind = transpose(rebin(wy,cy,cx))*nx + rebin(wx,cx,cy)
res = bigarr[ind]

I then wrote a little script to compare execution times, and I find
that if you are only extracting a small subsection (say, 500x10,000)
the index building method takes .04 seconds, compared to .11 seconds
for indexing bigarr twice. I'm really curious though why there's a
difference. Does it have something to do with using the asterix in
indexing? Is it just because it has two make two copies when I use
the first method? Since I'm extracting the same region either way,
I'd expect them to take about the same amount of time...
Jean H
2008-06-06 21:15:38 UTC
Permalink
Post by Conor
cx = n_elements(wx)
cy = n_elements(wy)
nx = n_elements(bigarr[*,0])
ind = transpose(rebin(wy,cy,cx))*nx + rebin(wx,cx,cy)
res = bigarr[ind]
You don't have to use nx.
res = bigarr[rebin(bigx,75,10000),rebin(bigy,10000,75)]

Note that you must have the same number of elements in every subscripted
dimension.

Read http://www.dfanning.com/code_tips/asterisk.html

Jean
Conor
2008-06-09 13:02:19 UTC
Permalink
Post by Jean H
Post by Conor
cx = n_elements(wx)
cy = n_elements(wy)
nx = n_elements(bigarr[*,0])
ind = transpose(rebin(wy,cy,cx))*nx + rebin(wx,cx,cy)
res = bigarr[ind]
You don't have to use nx.
res = bigarr[rebin(bigx,75,10000),rebin(bigy,10000,75)]
Note that you must have the same number of elements in every subscripted
dimension.
Readhttp://www.dfanning.com/code_tips/asterisk.html
Jean
On second thought, I'm now thoroughly confused. I don't understand
this bit:

res = bigarr[rebin(bigx,75,10000),rebin(bigy,10000,75)]

obviously rebin(bigx,75,10000) and rebin(bigy,10000,75) give you two
arrays with exactly opposite dimensions, so I don't understand how you
can index bigarr with these two arrays. Also, I'm not even sure what
IDL is doing when you index an array like that. My first thought was
that this was the equivelent of:

res = bigarr[ [rebin(bigx,75,10000),rebin(bigy,10000,75)] ]

But that obviously can't be the case because since they have different
dimensions, you can't concatenate the two arrays. So how does IDL
pull out an array index in your above example?
Jean H
2008-06-09 17:12:49 UTC
Permalink
Post by Conor
Post by Jean H
Post by Conor
cx = n_elements(wx)
cy = n_elements(wy)
nx = n_elements(bigarr[*,0])
ind = transpose(rebin(wy,cy,cx))*nx + rebin(wx,cx,cy)
res = bigarr[ind]
You don't have to use nx.
res = bigarr[rebin(bigx,75,10000),rebin(bigy,10000,75)]
Note that you must have the same number of elements in every subscripted
dimension.
Readhttp://www.dfanning.com/code_tips/asterisk.html
Jean
On second thought, I'm now thoroughly confused. I don't understand
res = bigarr[rebin(bigx,75,10000),rebin(bigy,10000,75)]
obviously rebin(bigx,75,10000) and rebin(bigy,10000,75) give you two
arrays with exactly opposite dimensions, so I don't understand how you
can index bigarr with these two arrays. Also, I'm not even sure what
IDL is doing when you index an array like that. My first thought was
res = bigarr[ [rebin(bigx,75,10000),rebin(bigy,10000,75)] ]
But that obviously can't be the case because since they have different
dimensions, you can't concatenate the two arrays. So how does IDL
pull out an array index in your above example?
Ah, I guess you are right... I was typing too fast (and the no-error
didn't warn me..)

it should have been:
IDL> help, bigarr[rebin(bigx,75,10000),transpose(rebin(bigy,10000,75))]

So, rebin(bigx,75,10000) replicates your 75 entries 10 000 times.
rebin(bigy,10000,75)) replicates the 10 000 entries 75 times. Now,
transpose(rebin(bigy,10000,75)), well, transposes it so the two
subscript now have the same dimensions. At last, for each corresponding
index in both subscripts, the value of bigArr is returned.

Was i posted was working because there are the same number of elements
in both subscripts... though the combination was wrong and therefore it
returns the wrong result..

Sorry for the confusion!
Jean
Jeremy Bailin
2008-06-07 13:12:26 UTC
Permalink
Post by Conor
So I want to pull out a small subsection from within a larger idl
bigarr = findgen(500,100000)
bigx = [indgen(50)+100,indgen(25)+200]
bigy = indgen(10000)+50000
res = bigarr[bigx,bigy]
In the particular case above, I would expect that this would be most
efficient:

res = [bigarr[100:149,50000:59999], bigarr[200:224,50000:59999]]

Of course, that's only going to work when you know a priori that
everything will be in a relatively small number of chunks.

-Jeremy.
Conor
2008-06-09 12:56:28 UTC
Permalink
Post by Jeremy Bailin
Post by Conor
So I want to pull out a small subsection from within a larger idl
bigarr = findgen(500,100000)
bigx = [indgen(50)+100,indgen(25)+200]
bigy = indgen(10000)+50000
res = bigarr[bigx,bigy]
In the particular case above, I would expect that this would be most
res = [bigarr[100:149,50000:59999], bigarr[200:224,50000:59999]]
Of course, that's only going to work when you know a priori that
everything will be in a relatively small number of chunks.
-Jeremy.
Interesting, thanks!
Heinz Stege
2008-06-09 15:34:29 UTC
Permalink
Post by Conor
So I want to pull out a small subsection from within a larger idl
bigarr = findgen(500,100000)
bigx = [indgen(50)+100,indgen(25)+200]
bigy = indgen(10000)+50000
res = bigarr[bigx,bigy]
% All array subscripts must be same size. Var = BIGARR
% Execution halted at: $MAIN$
You may want to try this command:
res = bigarr[bigx,bigy,0]

IDL accepts arrays of different size as subscripts, if at least one of
the subsripts is a scalar. Here the "0" addresses an "extra"
dimension. Since this extra dimension is of size 1, it is degenerated
in the result. Array res has two dimensions, 75x10000.

Heinz
Conor
2008-06-10 14:16:41 UTC
Permalink
Post by Heinz Stege
Post by Conor
So I want to pull out a small subsection from within a larger idl
bigarr = findgen(500,100000)
bigx = [indgen(50)+100,indgen(25)+200]
bigy = indgen(10000)+50000
res = bigarr[bigx,bigy]
% All array subscripts must be same size. Var = BIGARR
% Execution halted at: $MAIN$
res = bigarr[bigx,bigy,0]
IDL accepts arrays of different size as subscripts, if at least one of
the subsripts is a scalar. Here the "0" addresses an "extra"
dimension. Since this extra dimension is of size 1, it is degenerated
in the result. Array res has two dimensions, 75x10000.
Heinz
Hmm... this does work! It seems very strange though. Seems like a
rather arbitrary rule...

Loading...