| blob | aca1618559c6c7da5788f04422fee00b574fbf38 |
1 Efficient rendering of terrain elevation model
3 start with some top level cells
5 Thursday 20th Nov
6 terrain elev (errors)
7 naive interpolation
8 evaluation
9 views from google earth
10 qualitative
11 image data + info viewing / illumination
12 >1? - resampling
13 resampling together
15 Thurday 27th Nov
16 projection of imagery is not linear lon/lat
17 use gdal library to open file and get projection, and translate between projections
18 found another source of elevation data
19 3arcsec srtm with voids filled in using high resolution relief maps
20 1arcsec same format but made before the above with same maps
21 higher res, but SRTM is more accurate
22 elevation interpolation
23 tried bilinear and bicubic
24 quite a few anomolies
26 meeting
27 segmentation
28 [ ] comparing two regions in different datasets
29 [ ] get source from shadow guys
31 Monday 1st dec
32 using discontinuity measure with mid range IR
33 mir lights up rocks not in shadow, so discontinuity *(1-mir) could work
34 initially working with discontinuity measurement
35 saturation screws up with a lot of it
36 its then that i realised that of course shadow is never saturated
37 any saturated area can be discounted as non-shadow
38 shadows are generally dark throughout the channels
39 so multiply by (1- each channel value)
40 low (unsnowed) vegetation still shows up as shadow
41 is that because its shadows in the complexity of trees etc?
42 these areas have accurate SRTM data
43 perhaps we can use this to prove that they cannot be in shadow
44 characteristics of a cast shadow as travel away from light
45 area of light
46 area of self shadow - we can find this from the elevation data
47 area of cast shadow
48 area of light
49 therefore an area of light followed by an area of shadow
50 without self shadow in between is false
52 Thursday 4th Dec
53 raytracing
54 simple, per pixel towards light until hits terrain
55 edgetracing
56 edges of sun visibility projected into floor
57 lambertian shading
58 meeting
59 negative product might not work as with rgb, but quite good here
60 doesn't work well in non snowy areas
61 constraints transititions along light path of illuminated, self shadow, cast shadow
63 Monday 8th Dec
64 shadow classification isn't great using simple method
65 we could use some PAT techniques
66 features could be log chromaticity values
67 classifications are shadow/non-shadow
68 could restrict to only pixels negative-product classifies as shadow
69 interface
70 menu export->condition in selection
71 condition is channel and threshold (=0.5)
72 list of available channels
73 list of output channels
74 buttons to transfer between, and move up and down
75 generalised negative product
76 now product of powers of negative channels
77 can bias channels, so thermal IR has more effect at eliminating shadows than the others
79 Tuesday 9th Dec
80 [ ] improve shadow classification using self-shadow -> cast-shadow constraint
81 [ ] start terrain improvements
82 [ ] firstly apply
84 thursday 11th dec
85 [X] error function in terms of parameters
86 [ ] find derivitive (numerically)
87 [X] gradient descent
88 [X] threshold as another parameter or minimise threshold each step
89 [ ] could plot wavelength - importance to shadow classification
90 [X] whether throwing away channels would help - minimiser would solve
91 [ ] test that constraint equations hold on training data (russion relief maps)
92 [ ] take some lines in direction of sun, and do the plot
93 [ ] smooth out small variations between shadow types
94 [ ] ambient occlusion with hemispheres
95 [ ] "shape from shading on a cloudy day"
96 [ ] shape from shading - two images - intersection of cones
97 [ ] scatter plot between angle and colour in shadows
98 [ ] look good on report
99 [ ] lit review before christmas
101 Sunday 14th Dec
102 gradient descent
103 hold off on thresholding
104 error = number of incorrect classifications
105 as threshold goes from 0 to 1, crossing a pixel value
106 pixels in shadow go incorrect
107 pixels in light go correct
108 create a number of bins in range, e.g. for each slider value
109 single pass through pixels
110 if classified as shadow: +1 to bin
111 as threshold hits that value, error will increase
112 if classified as non-shadow: -1 from bin
113 as threshold hits that value, error will decrease
115 mon 15th dec
116 gradient descent comes up with something like:
117 threshold: 2.47319e-6
118 0.9, 1.2, 0.8, 2.6, 19.5, 14, 10.7, 8.5, 3.3
119 when adapted to larger terrain, optimal threshold is: 1.8692e-6
120 error ~6.32%
121 another gradient descent:
122 threshold: 0.000581442
123 0.1, 0.8, 0.9, 0.9, 15.8, 7.2, 6.2, 6.9, 2.8
124 error ~6.26%
126 tue 16th dec
127 added elevation channel, showing void
128 when change lambertian shading to light from top, and go high resolution
129 do voids appear darker, i.e. significantly steeper?
130 there's definitely some relationship, but its not entirely accurate
132 Thu 18th dec
133 lit review
134 about 6 papers per section
135 terrain
136 any attempt at estimation
137 Edwin did radar shape from shading
138 "francot" "chalaper" - surface integration
139 shape from shading from snow covered terrain
140 multispectral stuff
141 srtm paper?
142 yes
143 section on data sources
144 sensors etc
145 adv/disadv, accuracy
147 try gradient descent on big terrain area
148 check if it holds generally
149 scatter plots
150 normal angle to sun / value
151 normal z component / value
153 Thu 15th jan
154 shadow edge dictionary - low level (finding edges) -> higher level (meaning)
155 shadow rules
156 will ?[cg]ause? any problem
157 tree/graph/finite state automata
158 tree of allowed rules
159 Consistency - what to do when not
160 cost with moving
161 confidence in how good
162 minimal cost change
163 smoothness across
164 general rules - state machine
165 do sample - check consistent
166 add shading to cost
168 cost function
169 variables to include
170 change of height values (appropriately scaled)
171 (h-h0)^2 * certainty
172 shadow entrance constraints
173 difference of gradients to tangential
174 (-dh/dw - sin(elev_sun))^2
175 negative change in gradient
176 max(0,d^2h/dw^2)^2
177 elevation relative to transition exit, from elevation of sun
178 (h_exit+(w_enter-w_exit)*sin(elev_sun) - h)^2
179 total rate of change of gradient (smoothness) in both directions
180 (d^2h/dx^2)^2
181 (d^2h/dy^2)^2
182 shadow elevation constraint - below line between entrance and exit
183 (max(0, h - (h_exit*(w_exit-w) + h_enter*(w-w_enter)) / (w_exit-w_enter)))^2
184 lit pixels must be facing the sun
185 min(0, -dh/dw - sin(elev_sun))^2
186 angle between normals and shape-from-shading/occlusion normals
187 factoring in reliability of shape-from-shading (e.g. snow/rock)
188 requirements
189 quick access to a pixel's sun direction shadow entrance/exit
190 elevation field upsampled to match imagery?
192 Thur 22nd Jan
193 [ ] plot theta(N,L)/I to check shading is accurate
194 for shadow and non shadow
195 should look like cos in sun
196 tell us albedo (x=0), reflection when face on
197 [ ] what shape for shadow region?
198 integrate over L for occlusion
199 imagine hemisphere on surface (slanted)
200 the bottom part is cut off (horizon)
201 -- we integrate from one side to the other, across the slope
202 -- integrate the L.N across the points in the arc of a circle
203 albedo*L*int(0,pi, int(theta,pi, L.N dx) dy)
204 -- N is (0,0,1)
205 -- L is (cos(x)*sin(y), cos(y), sin(x)*sin(y))
206 -- L.N = sin(x)*sin(y)
207 albedo*L*int(0,pi, int(theta,pi, sin(x)*sin(y) dx) dy)
208 -- sin(y) doesn't depend on x
209 albedo*L*int(0,pi, sin(y)*int(theta,pi, sin(x) dx) dy)
210 -- int sin(x) dx = -cos(x)
211 albedo*L*int(0,pi, sin(y)*[-cos(pi)+cos(theta)] dy)
212 -- cos(pi) = -1
213 albedo*L*int(0,pi, sin(y)*[1+cos(theta)] dy)
214 -- [1+cos(theta)] doesn't depend on y
215 albedo*L*[1+cos(theta)]*int(0,pi, sin(y) dy)
216 albedo*L*[1+cos(theta)]*[-cos(pi) + cos(0)]
217 -- cos(pi) = -1, cos(0) = 1
218 albedo*L*[1+cos(theta)]*2
219 -- rearrange
220 2*albedo*L*(1+cos(theta))
222 Monday 26th Jan
223 so to implement the above cost function, we need at each pixel:
224 current elevation (turn into height map)
225 gradient in light directions
226 rate of change of gradient in x, y, light direction
227 whether classified as shadow, or lit
228 whether on a boundary of shadow and light
229 sun elevation and direction
230 if in shadow:
231 distance to and current elevation at shadow entrance and exit in light direction
233 Tue 27th Jan
234 now we have shadow depth channel in each direction
235 new channel to optimise elevation
236 must be able to take multiple datasets
237 also adjusts corrected elevation values in DEM
238 inputs
239 previous elevation (internal)
240 shadow classification, shadow depths
242 Thur 29th Jan
243 We really need to know distances accurately
244 coordinate spaces
245 geographical
246 can calculate short distances easily, given radius
247 can convert to texture space
248 texture space
249 can convert to geographical
250 can't directly calculate distances
251 ideally need to optimise in a geographical texture space
252 resample shadow
253 19:13 - tried with roughness, variance, lit facing and travision elev
254 hard to get coefficients right so that it does the smoothing
255 better result when learning rate is constant
257 Wed 4th Feb
258 earlier in the week got all channels into geographical space, resampled
259 on loading. this should make it more convenient to optimise using multiple
260 data sets later on.
261 now updates digital elevation model with optimisation
262 now need to get optimisation improved!
264 Thur 5th Feb
265 [ ] graphs lit/nonlit
266 [ ] project title
267 "shadow segmentation"
268 "satellite imagery"
269 "multispectral"
270 "elevation correction"
271 "void filling"
272 "correction of terrain elevation data using shadow segmentation of multispectral imagery"
273 "fine scale correction of terrain elevation data using terrain imagery"
274 [ ] statement of ethics talk by alistair
275 [ ] elevation correction
276 [ ] tackling wall problem
277 [ ] intermediate smoothing
278 [ ] start at low resolution, and increase resolution as we get closer
279 [ ] moving blocks of pixels at a time
280 [ ] when moving pixels, move others in range, like blender does
282 Mon 9th Feb
283 used falloff function when editing verticies
284 gaussian is very very slow
285 linear is much faster, still slow, but converges faster than before
286 massive dip on each side because forgot to max(0,x)
287 overshoots because no constraint to keep shadow areas below shadow line
288 add shadow line constraint - not working yet, pushing terrain down
290 Tue 10th Feb
291 landsat 5 data now loads
292 need to make it possible to work on multiple datasets
293 colour map widget
294 [X] add groups to ColourMapWidget
295 simple addGroupSeparator(const QString& name)
296 [X] global channel manager for optimisation and terrain channels
297 [X] elevation optimisation of multiple image datasets
298 [ ] fix export to work on multiple datasets
300 Thur 12th Feb
301 works on multiple datasets, slowly
302 new problem, not updating textures during mid-optimization refresh
303 optimization gui
304 [ ] for each weight, also have constant/increase/decrease option
305 [ ] setting for dh
306 [ ] setting for linear falloff limit
307 DEM gui
308 [ ] keep DEM in memory when switching
309 [ ] save corrected values
310 [X] compare DEMs
311 [ ] simplify - corrected values are separate DEM (primary, secondary)
312 [X] select primary and secondary DEM
313 [ ] void-fill performed on primary DEM
314 [ ] corrections written to secondary DEM
315 [ ] initialised with new secondary DEM copied from primary
316 [ ] new elevation data
317 [ ] select custom resolution
318 [ ] copy from another DEM
319 [X] select to display primary or secondary DEM, OR...
320 [X] compare primary, secondary DEM
321 [O] slider with partial updating of textures too (periodically)
322 [ ] channel to compare primary and secondary
324 Mon 16th Feb
325 need to get on with write up
326 added DEM comparison channel - allows quantitative comparison of std deviation
327 John Clark in CRY L10 Non standard cracking
328 when optimizing, make problem harder to get better results
329 in context of perceptron problem
330 move boundary of punishment so values are further from border and
331 less likely to cross it again soon
332 may be very unlikely to solve strict problem but gives us better
333 solution to our problem in the process
334 Negative product optimization implications
335 instead of error being number of incorrect pixels
336 error is sum distance on wrong side of boundary
337 e.g. if np=0.2, shadow, th=0.3, err=th/np=3/2
338 [ ] could also try simulated anealing
340 thu 19th feb (02:10)
341 got more image datasets with much lower solar elevation
342 very pronounced shadows (a bit too big in fact!)
343 haven't tried optimizing yet
344 ideally need to get a good set of different solar positions
345 plot solar positions in 2d plot to show variation
346 it also occurred to me that for the purposes of testing the algorithm
347 could generate shadow classification directly from raytracing with
348 some specially chosen sun direction (pretending we've done the
349 shadow classification correctly on some fictional image dataset)
350 e.g. to get a higher azimuth than 160deg, its always morning atm
352 fix list
353 [ ] seems - not important
354 [ ] when creating texture for region, don't up sample, select larger area
356 todo
357 [X] proper texturing
358 [X] selection of channels to display
359 [ ] more sensible sampling of elevation data
360 [X] edges of box should interpolate
361 [X] inner vertices should be sensible
362 [ ] srtm
363 [X] basic linear interpolation
364 [X] spline interpolation
365 [X] gui to toggle {flat, unprocessed, corrected -> refined}
366 [ ] self shadow lines
367 [ ] different colours for ridges and valleys
368 [ ] interaction
369 [X] focus extended altitude
370 [X] mouse click -> lon lat transformation
371 [ ] terrain grab while dragging
372 [X] selection of region
373 [ ] region image preview (full resolution)
374 [ ] multiple viewports
375 [ ] interaction (observers) can be locked together
376 [ ] data view can be locked together
377 [ ] image processing
378 [ ] should be able to add processing bands to those on the colour mapper
379 [X] input bands should be derivable from digital elevation model
380 e.g. could do calculations in world space instead of texture space
381 [X] get texture space normal at a pixel
382 [X] get texture space lighting direction at a pixel
383 [X] get elevation at a pixel
384 [X] get geographical coordinate at a pixel
385 examples
386 [X] shadow from DEM
387 [X] lambertian shading (automatic self shadow detection)
388 [ ] combination of the two (pixel product)
389 [ ] chromaticity based on lambertian shading shows colour
390 [ ] shadow detection
391 [X] in region, detect shadow areas
392 [X] from imagery
393 [X] from elevation data using raytracing
394 [ ] vectorisation
395 [ ] border and highlight in both views
397 texturing
398 problems
399 textures are BIG
400 8401 x 7461 - 63MB x6
401 4201 x 3731 - 16MB x2
402 16801 x 14921 - 250MB x1
403 660MB
404 solution
405 downsample entire image to a reasonable size
406 1/8 x 1/8 = 1/64th size = ~10MB
407 or use the thumbnail in each image
408 when a portion of the image is desired in higher detail
409 reload portion of the image at full resolution
410 1/8 x 1/8 = 1/64th size of image
411 when a portion of the image is no longer being used
412 discard of it
414 class tcBigImage
415 width, height
416 thumbnail texture
417 rows, cols
418 cells[rows][cols]
419 texture
420 image data
422 image processing
423 image data should be made up of channels
424 some channels are source bands from GeoTIFF files
425 some are derived from other channels
426 tcChannelManager
427 manages a set of channels that are referred to in a colour map widget
428 tcChannel
429 functionality for keeping data around so as not to have to reload
430 after creating GL texture
431 results can be floating point, bytes, bits
432 histograms and various plots based on the pixel data
433 data changed event triggers invalidation of textures
434 and invalidation of channels derived from this channel
435 tcChannelConstant : tcImageChannel
436 constant colour - e.g. black (for a null channel to avoid clicking rgb each time)
437 tcChannelFile : tcImageChannel
438 data from a GeoTIFF file
439 tcChannelProcess
440 acts on a number of input channels to produce a number of output channels
441 can be configured with a gui
442 tcChannelProcessChromaticity : tcChannelProcess
443 divide a set of channels by another channel or a value
444 out: float
445 tcChannelProcessIlluminantInvariant : tcChannelProcess
446 input log chromaticities
447 can configure illuminant direction
448 if two dimentions, use a dial
449 for > 2 dimentions, select a shadow / non shadow adjascent bit of image
450 or select multiple such known shadow changes and average
451 find variation vector from this
452 easy to reuse this vector
453 tcChannelProcessIlluminantDiscontinuity : tcChannelProcess
454 see cic13
455 shows discontinuity between adjascent pixels in x or y direction
456 i.e. you might show two of these in different colour channels
458 renderCell
459 determine if we need to subdivide quality in this cell
460 if we're close, then we do
461 find distance from each corner
462 use distance[i] < max(dist(corner[i], corner[j]))*factor
463 if we should subdivide
464 renderCell on each of the 4 subdivisions
465 determine the quality at this level
466 find distance to center
467 apply magic function to find detail level
468 render the cell in this detail level