[Yt-svn] yt: 23 new changesets
hg at spacepope.org
hg at spacepope.org
Wed Oct 27 21:24:40 PDT 2010
hg Repository: yt
details: yt/rev/d0295e86e753
changeset: 3468:d0295e86e753
user: Matthew Turk <matthewturk at gmail.com>
date:
Mon Oct 25 16:38:01 2010 -0700
description:
ART now instantiates
hg Repository: yt
details: yt/rev/c08d727a826a
changeset: 3469:c08d727a826a
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Mon Oct 25 16:58:49 2010 -0700
description:
Fixed indexing in the read_art_vars in fortran_reader. Now we're at indexing consistent with global iOctCh index, and not idc
hg Repository: yt
details: yt/rev/7b5543b1c557
changeset: 3470:7b5543b1c557
user: Matthew Turk <matthewturk at gmail.com>
date:
Mon Oct 25 17:04:13 2010 -0700
description:
Minor fixes
hg Repository: yt
details: yt/rev/1706d0bdb1a8
changeset: 3471:1706d0bdb1a8
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 10:56:28 2010 -0700
description:
Fixes to add a root grid object as well as to get the indices correct for
the levels.
hg Repository: yt
details: yt/rev/d83c2166b211
changeset: 3472:d83c2166b211
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 09:58:03 2010 -0700
description:
IO reader update
hg Repository: yt
details: yt/rev/21866e57e291
changeset: 3473:21866e57e291
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 11:51:50 2010 -0700
description:
read_art_vars works now
hg Repository: yt
details: yt/rev/589c2beebd7e
changeset: 3474:589c2beebd7e
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 12:19:28 2010 -0700
description:
read_art_vars (root grid IO) works
hg Repository: yt
details: yt/rev/72b5dc41d0c2
changeset: 3475:72b5dc41d0c2
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 13:42:06 2010 -0700
description:
Sketch of new IO handler for ART
hg Repository: yt
details: yt/rev/f71dec35a00c
changeset: 3476:f71dec35a00c
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 13:44:59 2010 -0700
description:
Adding back in the read_art_grid function
hg Repository: yt
details: yt/rev/542e470438fa
changeset: 3477:542e470438fa
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 15:37:40 2010 -0700
description:
Updated read_art_grid to use read_art_vars or the read_art_root_vars
hg Repository: yt
details: yt/rev/b2fa357ca8c4
changeset: 3478:b2fa357ca8c4
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 15:47:12 2010 -0700
description:
Hooking read_art_grid up to read_art_(root_)_vars
hg Repository: yt
details: yt/rev/3f5d79a53049
changeset: 3479:3f5d79a53049
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 15:52:03 2010 -0700
description:
Hooking read_art_grid up to read_art_(root_)_vars (again)
hg Repository: yt
details: yt/rev/f026fe2de455
changeset: 3480:f026fe2de455
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 16:08:24 2010 -0700
description:
Fixed most -- but not quite all -- the issues with left/right edges.
hg Repository: yt
details: yt/rev/66d8cfb1d018
changeset: 3481:66d8cfb1d018
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 16:30:56 2010 -0700
description:
Fixes to get the IO handler to instantiate properly
hg Repository: yt
details: yt/rev/b833ed10dd2d
changeset: 3482:b833ed10dd2d
user: Christopher Erick Moody <cemoody at ucsc.edu>
date:
Tue Oct 26 16:40:13 2010 -0700
description:
Added field_list
hg Repository: yt
details: yt/rev/a9b8d8ce0c25
changeset: 3483:a9b8d8ce0c25
user: Matthew Turk <matthewturk at gmail.com>
date:
Tue Oct 26 17:55:29 2010 -0700
description:
More fixes for ART; can successfully read (Without error, but wrong units!) the
non-root grids.
hg Repository: yt
details: yt/rev/1a73cbcadf16
changeset: 3484:1a73cbcadf16
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 00:27:29 2010 -0700
description:
changed to using absolute offsets to each individual level's records
hg Repository: yt
details: yt/rev/d5ec2a53bca4
changeset: 3485:d5ec2a53bca4
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 00:45:38 2010 -0700
description:
Added an assert; when this runs correctly, we should have the data.
hg Repository: yt
details: yt/rev/2746e31f3339
changeset: 3486:2746e31f3339
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 09:41:16 2010 -0700
description:
Adding in reading of 'var' to get the record size up to 56 bytes
hg Repository: yt
details: yt/rev/fbf6a6c6e4a9
changeset: 3487:fbf6a6c6e4a9
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 11:37:27 2010 -0700
description:
Child grids work; root grid has been bypassed. IO is incredibly slow. Next
phase will be a pre-loadable set of arrays.
hg Repository: yt
details: yt/rev/0953dc407f99
changeset: 3488:0953dc407f99
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 12:32:43 2010 -0700
description:
Moving to caching the data level-by-level (with no automatic release). Works
for L>0.
hg Repository: yt
details: yt/rev/54b39d70b365
changeset: 3489:54b39d70b365
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 15:30:56 2010 -0700
description:
Fixes for fortran ordering and root grid. Projections of density can now be
made, but units are tricky.
hg Repository: yt
details: yt/rev/3ad9c5313cc6
changeset: 3490:3ad9c5313cc6
user: Matthew Turk <matthewturk at gmail.com>
date:
Wed Oct 27 21:23:07 2010 -0700
description:
Removing cevart and adding the art subpackage
diffstat:
yt/frontends/art/data_structures.py | 59 ++-
yt/frontends/art/io.py | 132 +++++----
yt/frontends/cevart/__init__.py | 29 --
yt/frontends/cevart/api.py | 42 ---
yt/frontends/cevart/ceverinoreader.py | 291 --------------------
yt/frontends/cevart/data_structures.py | 405 -----------------------------
yt/frontends/cevart/fields.py | 61 ----
yt/frontends/cevart/io.py | 59 ----
yt/frontends/cevart/setup.py | 10 -
yt/frontends/setup.py | 2 +-
yt/utilities/_amr_utils/fortran_reader.pyx | 211 ++++++++------
11 files changed, 230 insertions(+), 1071 deletions(-)
diffs (truncated from 1477 to 300 lines):
diff -r e7a6711f94ea -r 3ad9c5313cc6 yt/frontends/art/data_structures.py
--- a/yt/frontends/art/data_structures.py Tue Oct 26 11:50:30 2010 -0700
+++ b/yt/frontends/art/data_structures.py Wed Oct 27 21:23:07 2010 -0700
@@ -38,6 +38,7 @@
from yt.data_objects.static_output import \
StaticOutput
from .fields import ARTFieldContainer
+from .fields import add_field
from yt.utilities.definitions import \
mpc_conversion
from yt.utilities.io_handler import \
@@ -119,7 +120,10 @@
pass
def _detect_fields(self):
- self.field_list = self.tree_proxy.field_names[:]
+ self.field_list = [ 'Density','Gas_Energy',
+ 'x-momentum','y-momentum','z-momentum',
+ 'Pressure','Gamma','Total_Energy','Potential_New'
+ 'Potential_Old']
def _setup_classes(self):
dd = self._get_data_reader_dict()
@@ -136,19 +140,24 @@
self.pf.parameter_filename, self.pf.child_grid_offset,
self.pf.min_level, self.pf.max_level, self.pf.nhydro_vars,
self.pf.level_info)
- num_ogrids = sum(level_info) + self.pf.iOctFree
+ self.pf.level_info = na.array(self.pf.level_info)
+ num_ogrids = sum(self.pf.level_info) + self.pf.iOctFree
ogrid_left_indices = na.zeros((num_ogrids,3), dtype='int64') - 999
ogrid_levels = na.zeros(num_ogrids, dtype='int64')
ogrid_file_locations = na.zeros((num_ogrids,6), dtype='int64')
ogrid_parents = na.zeros(num_ogrids, dtype="int64")
ochild_masks = na.zeros((num_ogrids, 8), dtype='int64').ravel()
- amr_utils.read_art_tree(self.pf.parameter_filename,
+ self.pf.level_offsets = amr_utils.read_art_tree(
+ self.pf.parameter_filename,
self.pf.child_grid_offset,
self.pf.min_level, self.pf.max_level,
ogrid_left_indices, ogrid_levels,
- ogrid_parents, ochild_masks)
+ ogrid_parents, ochild_masks,
+ ogrid_file_locations)
+ self.pf.level_offsets = na.array(self.pf.level_offsets, dtype='int64')
+ self.pf.level_offsets[0] = self.pf.root_grid_offset
ochild_masks.reshape((num_ogrids, 8), order="F")
- ogrid_levels[ogrid_left_indices[:,0] == -999] = 0
+ ogrid_levels[ogrid_left_indices[:,0] == -999] = -1
# This bit of code comes from Chris, and I'm still not sure I have a
# handle on what it does.
final_indices = ogrid_left_indices[na.where(ogrid_levels==self.pf.max_level)[0]]
@@ -156,11 +165,20 @@
for level in xrange(self.pf.max_level*2)]
root_level = self.pf.max_level+na.where(na.logical_not(divisible))[0][0]
ogrid_dimension = na.zeros(final_indices.shape,dtype='int')+2
- ogrid_left_indices = ogrid_left_indices/2**(root_level - ogrid_levels[:,None]) - 1
+ ogrid_left_indices = ogrid_left_indices/2**(root_level - ogrid_levels[:,None] - 1) - 1
+
# Now we can rescale
- self.proto_grids = []
- for level in xrange(len(level_info)):
- if level_info[level] == 0:
+ root_psg = _ramses_reader.ProtoSubgrid(
+ na.zeros(3, dtype='int64'), # left index of PSG
+ self.pf.domain_dimensions, # dim of PSG
+ na.zeros((1,3), dtype='int64'), # left edges of grids
+ self.pf.domain_dimensions[None,:], # right edges of grids
+ self.pf.domain_dimensions[None,:], # dims of grids
+ na.zeros((1,6), dtype='int64') # empty
+ )
+ self.proto_grids = [[root_psg],]
+ for level in xrange(1, len(self.pf.level_info)):
+ if self.pf.level_info[level] == 0:
self.proto_grids.append([])
continue
ggi = (ogrid_levels == level).ravel()
@@ -291,8 +309,9 @@
for g in grid_list:
fl = g.grid_file_locations
props = g.get_properties()
- self.grid_left_edge[gi,:] = props[0,:] / (2.0**(level+1))
- self.grid_right_edge[gi,:] = props[1,:] / (2.0**(level+1))
+ dds = ((2**level) * self.pf.domain_dimensions).astype("float64")
+ self.grid_left_edge[gi,:] = props[0,:] / dds
+ self.grid_right_edge[gi,:] = props[1,:] / dds
self.grid_dimensions[gi,:] = props[2,:]
self.grid_levels[gi,:] = level
grids.append(self.grid(gi, self, level, fl, props[0,:]))
@@ -336,15 +355,18 @@
self.derived_field_list = []
def _setup_data_io(self):
- pass
- #self.io = io_registry[self.data_style](self.tree_proxy)
+ self.io = io_registry[self.data_style](
+ self.pf.parameter_filename,
+ self.pf.nhydro_vars,
+ self.pf.level_info,
+ self.pf.level_offsets)
class ARTStaticOutput(StaticOutput):
_hierarchy_class = ARTHierarchy
_fieldinfo_class = ARTFieldContainer
_handle = None
- def __init__(self, filename, data_style='ramses',
+ def __init__(self, filename, data_style='art',
storage_filename = None):
StaticOutput.__init__(self, filename, data_style)
self.storage_filename = storage_filename
@@ -353,9 +375,8 @@
self.current_time = 0.0
self.dimensionality = 3
self.refine_by = 2
- self.parameters["HydroMethod"] = 'ramses'
+ self.parameters["HydroMethod"] = 'art'
self.parameters["Time"] = 1. # default unit is 1...
- self.fh = open(storage_filename,'rb') #used by the io
def __repr__(self):
return self.basename.rsplit(".", 1)[0]
@@ -467,9 +488,11 @@
# This is not the same as the number of Octs.
self.domain_dimensions = na.ones(3, dtype='int64') * int(2**est)
+ self.root_grid_mask_offset = f.tell()
+ _skip_record(f) # iOctCh
self.root_grid_offset = f.tell()
- for to_skip in ['iOctCh', 'hvar', 'var']:
- _skip_record(f)
+ _skip_record(f) # hvar
+ _skip_record(f) # var
self.iOctFree, self.nOct = struct.unpack('>ii', _read_record(f))
self.child_grid_offset = f.tell()
diff -r e7a6711f94ea -r 3ad9c5313cc6 yt/frontends/art/io.py
--- a/yt/frontends/art/io.py Tue Oct 26 11:50:30 2010 -0700
+++ b/yt/frontends/art/io.py Wed Oct 27 21:23:07 2010 -0700
@@ -24,74 +24,84 @@
"""
import numpy as na
+import struct
from yt.utilities.io_handler import \
BaseIOHandler
+import numpy as na
+
+from yt.utilities.io_handler import \
+ BaseIOHandler
+import yt.utilities.amr_utils as au
class IOHandlerART(BaseIOHandler):
_data_style = "art"
- #at which position in the child record does the field occur
- field_dict = {'gas density':0}
-
- def __init__(self, art_tree, *args, **kwargs):
+
+ def __init__(self, filename, nhydro_vars, level_info, level_offsets,
+ *args, **kwargs):
BaseIOHandler.__init__(self, *args, **kwargs)
+ self.filename = filename
+ self.nhydro_vars = nhydro_vars
+ self.level_info = level_info
+ self.level_offsets = level_offsets
+ self.level_data = {}
+ def preload_level(self, level):
+ if level in self.level_data: return
+ if level == 0:
+ self.preload_root_level()
+ return
+ f = open(self.filename, 'rb')
+ f.seek(self.level_offsets[level])
+ ncells = 8*self.level_info[level]
+ nvals = ncells * (self.nhydro_vars + 6) # 2 vars, 2 pads
+ arr = na.fromfile(f, dtype='>f', count=nvals)
+ arr = arr.reshape((self.nhydro_vars+6, ncells), order="F")
+ arr = arr[3:-3,:].astype("float64")
+ self.level_data[level] = arr
- def _read_data_set(self,grid,field):
- fn = grid.pf.storage_filename
- # because of reuse promote this handler to pf class?
- min_level = grid.pf.min_level
- max_level = grid.pf.max_level
- nhydro_vars = grid.pf.nhydro_vars
- grid_level = grid.Level
- #grid_idc = (grid.id-1)*8+grid.pf.ncell+1 (verbatim analysis_ART)
- grid_idc = (grid.id)*8+grid.pf.ncell #this gives only the first cell of an oct
- header_offset = grid.pf.child_grid_offset
+ def preload_root_level(self):
+ f = open(self.filename, 'rb')
+ f.seek(self.level_offsets[0] + 4) # Ditch the header
+ ncells = self.level_info[0]
+ nvals = ncells * (self.nhydro_vars) # 0 vars, 0 pads
+ arr = na.fromfile(f, dtype='>f', count=nvals).astype("float64")
+ arr = arr.reshape((self.nhydro_vars, ncells), order="F")
+ self.level_data[0] = arr
- record_size = 2+1+nhydro_vars+3 #two pads, idc, hydro vars, 3 vars
- #go past the previous levels
- # skip the header
- offset = (header_offset
- #per level:
- # 4 bytes per integer, float or pad
- # first section has three integers + 2 pads
- + 4*5*(grid_level)
- # second section has 13 floats +2 pads per oct
- # (level_info) is the number of octs per level
- + 4*15*sum(level_info[:grid_level])
- # after the oct section is the child section.
- # there are 2 pads, 1 integer child ID (idc)
- # then #nhydro_vars of floats + 3 vars
- #there 8 times as many children as octs
- + 4*(2+1+nhydro_vars+3)*8*sum(level_info[:grid_level-1]))
- pdb.set_trace()
- fh = open(fn,'rb')
- fh.seek(offset)
- dtype='>i,'+'>i,'+'>%df,'%(nhydro_vars+3)+'>i'
- first_idc = na.fromfile(fh,dtype=dtype,count=1)[0][1]
- #this is the first idc of our section. we can guess
- # how many records away our idc is since idc increments
- # by 1 for almost every record. because idc can increase
- # by more than 1 sometimes, we will always overestimate
- # the number of records we should skip ahead. so work
- # backwards from our guess
- seek_guess = (grid_idc - first_idc)*record_size*4
- fh.seek(offset+seek_guess)
- while True:
- record = na.fromfile(fh,dtype=dtype,count=1)[0]
- # make sure that the pad bytes line up
- # otherwise we may have a phase offset or have stepped out
- # our section
- assert record[0]==record[-1]
- # we better have overestimated the idc
- assert record[1]>= grid_idc
- if record[1] == grid_idc:
- fh.close()
- return record[2][field_dict[field]]
- else:
- # in the next iteration we'll read the previous record
- # so rewind the last read, then rewind one further record
- fh.seek(fh.tell()-2*record_size)
- fh.close()
-
-
\ No newline at end of file
+ def clear_level(self, level):
+ self.level_data.pop(level, None)
+
+ def _read_data_set(self, grid, field):
+ pf = grid.pf
+ field_id = grid.pf.h.field_list.index(field)
+ if grid.Level == 0: # We only have one root grid
+ self.preload_level(0)
+ tr = self.level_data[0][field_id,:].reshape(
+ pf.domain_dimensions, order="F")
+ return tr.swapaxes(0, 2)
+ tr = na.zeros(grid.ActiveDimensions, dtype='float64')
+ filled = na.zeros(grid.ActiveDimensions, dtype='int32')
+ to_fill = grid.ActiveDimensions.prod()
+ grids = [grid]
+ l_delta = 0
+ while to_fill > 0 and len(grids) > 0:
+ next_grids = []
+ for g in grids:
+ self.preload_level(g.Level)
+ #print "Filling %s from %s (%s)" % (grid, g, g.Level)
+ to_fill -= au.read_art_grid(field_id,
+ grid.get_global_startindex(), grid.ActiveDimensions,
+ tr, filled, self.level_data[g.Level],
+ g.Level, 2**l_delta, g.locations)
+ next_grids += g.Parent
+ grids = next_grids
+ l_delta += 1
+ return tr
+
+ def _read_data_slice(self, grid, field, axis, coord):
+ sl = [slice(None), slice(None), slice(None)]
+ sl[axis] = slice(coord, coord + 1)
+ return self._read_data_set(grid, field)[sl]
+
+
diff -r e7a6711f94ea -r 3ad9c5313cc6 yt/frontends/cevart/__init__.py
--- a/yt/frontends/cevart/__init__.py Tue Oct 26 11:50:30 2010 -0700
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,29 +0,0 @@
-"""
-API for yt.frontends.ramses
-
-Author: Matthew Turk <matthewturk at gmail.com>
-Affiliation: UCSD
-Author: J.S. Oishi <jsoishi at gmail.com>
-Affiliation: KIPAC/SLAC/Stanford
-Author: Britton Smith <brittonsmith at gmail.com>
-Affiliation: MSU
-Homepage: http://yt.enzotools.org/
More information about the yt-svn
mailing list