[yt-users] Dark Sky Simulations, analysis gone wrong.

[Nathan Goldbaum]

Sorry, I'm confused.

In your first email you said you were having issues with the dark skies
simulation, but it seems you're trying to load it using the Gadget frontend
somehow? Or is this a different Gadget dataset?

As far as I'm aware, Dark Skies is available in the SDF output format, or
streaming over HTTP via thingking.

If it's just a big Gadget dataset, the issue is that it's too big to load
given the memory available on your machine. Right now you need enough
memory to store the Morton indices for all the particles (one int64 per
particle). We are actively working on improving this situation by making yt
use compressed bitmaps to store the Morton indices, but that work won't be
done for several months at least.

You should be able to stream the dark skies data over HTTP though without
having a ton of memory, there were some clever tricks applied to get around
the memory limit I described above.

The reason I was asking about the field_list for dark skies is because I'd
like to see what fields are available. The error message you saw indicates
that one or more of the fields necessary to calculate the radial velocity
is not available (or yt is not aware of an alias from a field name on disk
to yt's internal field name).

On Sat, Jul 1, 2017 at 6:46 PM ALEXANDROS TSOUROS <ph4602 at edu.physics.uoc.gr>
wrote:

> Thanks for your prompt reply!
>
> So this is another problem. I'm using a 64-bit python installation, yet
> the output of 'print ds.file_list' is
>
> "
> (...)
> Traceback (most recent call last):
>    File "ds.py", line 5, in <module>
>      print ds.field_list
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/data_objects/static_output.py",
> line 545, in field_list
>      return self.index.field_list
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/data_objects/static_output.py",
> line 503, in index
>      self, dataset_type=self.dataset_type)
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/geometry/particle_geometry_handler.py",
> line 39, in __init__
>      super(ParticleIndex, self).__init__(ds, dataset_type)
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/geometry/geometry_handler.py",
> line 50, in __init__
>      self._setup_geometry()
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/geometry/particle_geometry_handler.py",
> line 50, in _setup_geometry
>      self._initialize_particle_handler()
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/geometry/particle_geometry_handler.py",
> line 104, in _initialize_particle_handler
>      self._initialize_indices()
>    File
>
> "/home/atsouros/gadget2/yt/yt-conda/src/yt-git/yt/geometry/particle_geometry_handler.py",
> line 126, in _initialize_indices
>      morton = np.empty(self.total_particles, dtype="uint64")
> MemoryError
> "
>
> I would assume that the problem is that it's running out of ram. In
> general, I am able to produce the examples provided by Dark Sky, so I
> can in general load data from the dataset. But I cannot use 'print
> ds.field_list'.
>
> TL;DR
>
> The output is 'MemoryError'.
>
>
>
> On 2017-07-02 01:34, Nathan Goldbaum wrote:
> > What's the output of "print(ds.field_list)" with the loaded dataset?
> >
> > On Sat, Jul 1, 2017 at 4:21 PM ALEXANDROS TSOUROS
> > <ph4602 at edu.physics.uoc.gr> wrote:
> >
> >> Hello all!
> >>
> >> I am quite new to yt, so please bear with my ignorance!
> >>
> >> I need to perform analysis on the open data of the Dark Sky
> >> Simulation
> >> (http://darksky.slac.stanford.edu). In particular I need to create 1
> >> dimensional profiles of the radial velocity of dark matter particles
> >> around halos.
> >>
> >> Attached, you'll find my attempt at producing such a profile.
> >>
> >> When I do this on a GADGET2 simulation I've run, I can use the
> >> 'radial_velocity' field without a problem. However, this script,
> >> where
> >> the data is that of the Dark Sky simulation, produces the following
> >> output :
> >>
> >> '
> >> (...)
> >> yt.utilities.exceptions.YTFieldNotFound: Could not find field
> >> '('all',
> >> 'radial_velocity')' in ds14_a_1.0000.'
> >>
> >> However, as I understand it, the field 'radial_velocity' is native
> >> to
> >> yt. How can I surpass this problem?
> >>
> >> Any help will be appreciated.
> >>
> >> Thank you very much for your time,
> >> Tsouros Alexandros
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20170701/5946d673/attachment.htm>