[yt-dev] Good benchmark?

[Matthew Turk]

Hey Sam,

On Fri, Mar 1, 2013 at 2:38 PM, Sam Skillman <samskillman at gmail.com> wrote:
> I think in terms of providing a test for the cluster, the benchmark should
> not require the shipment of data.  For the benchmarking of yt for yt's own
> sake, then we do need to test the IO performance.

I'm not really sure that we want to avoid IO...

>
> I think it'd be great to make an in-memory dataset that can then be used for
> testing performance.  For example, set up a bunch of refined spheres and
> then project/slice/profile/render/halo find?

Perhaps, but will our current IC generators work at really, really big
scale?  I didn't think they would.  Plus it's hard to get them to
subdivide -- you end up with ltos of nested grids.

>
> Sam
>
>
> On Fri, Mar 1, 2013 at 12:31 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Hi Jeff,
>>
>> Sure.  I got asked for a standard yt performance benchmark to be run
>> to figure out performance of an analysis cluster and evaluate its
>> readiness.  :)  I think it's safe to sya that we should be pushing
>> things like IO, memory capacity and communication performance.
>>
>> Adding such a set of scripts (and their results!) would be very useful
>> going forward, along with a little description of why we chose the
>> routines we did.
>>
>> -Matt
>>
>> On Fri, Mar 1, 2013 at 2:17 PM, j s oishi <jsoishi at gmail.com> wrote:
>> > Matt,
>> >
>> > It might be helpful if you shared some more details of this particular
>> > benchmarking exercise, if you can. It would be helpful for making sure
>> > we present the most usefil information to the people asking for it as
>> > well as info that is useful to us going forwards.
>> >
>> > j
>> >
>> > On Fri, Mar 1, 2013 at 2:05 PM, Matthew Turk <matthewturk at gmail.com>
>> > wrote:
>> >> Definitely, yes.  For what it's worth, the specific use case I have in
>> >> mind is for performance testing a system, so I think scalability will
>> >> be important.
>> >>
>> >> On Fri, Mar 1, 2013 at 2:01 PM, Cameron Hummels <chummels at gmail.com>
>> >> wrote:
>> >>> It might be worth showing how well this works on a single proc as well
>> >>> as
>> >>> how well it works using parallel mode on a few different numbers of
>> >>> processors.
>> >>>
>> >>>
>> >>> On Fri, Mar 1, 2013 at 1:57 PM, Christopher Moody
>> >>> <chrisemoody at gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> Perhaps a benchmark for simply calculating a derived field? This
>> >>>> would
>> >>>> then profile a much smaller codebase.
>> >>>>
>> >>>> Also benchmarking projections/fields for AMR/octree/particles of
>> >>>> similar
>> >>>> resolutions would be pretty cool.
>> >>>>
>> >>>> chris
>> >>>>
>> >>>>
>> >>>> On Fri, Mar 1, 2013 at 10:53 AM, Matthew Turk <matthewturk at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Okay, I like that idea.  So a unified script with timing for in each
>> >>>>> section might include:
>> >>>>>
>> >>>>>  * Halo profiling
>> >>>>>  * Global projection
>> >>>>>  * Global profiles
>> >>>>>  * Global VR (CTF and OffAxisProj)
>> >>>>>
>> >>>>> I think we probably want a large unigrid and a large AMR dataset to
>> >>>>> run these on, too.
>> >>>>>
>> >>>>> On Fri, Mar 1, 2013 at 1:50 PM, Nathan Goldbaum
>> >>>>> <goldbaum at ucolick.org>
>> >>>>> wrote:
>> >>>>> > Probably a good idea to try some off axis projections or simple
>> >>>>> > volume
>> >>>>> > renderings to test the VR code.
>> >>>>> >
>> >>>>> >
>> >>>>> > On Fri, Mar 1, 2013 at 10:48 AM, Matthew Turk
>> >>>>> > <matthewturk at gmail.com>
>> >>>>> > wrote:
>> >>>>> >>
>> >>>>> >> Hi all,
>> >>>>> >>
>> >>>>> >> I got an email asking for a benchmark of yt.  I think this is
>> >>>>> >> very,
>> >>>>> >> very valuable to have going forward.  I was wondering if anyone
>> >>>>> >> had
>> >>>>> >> any suggestions?
>> >>>>> >>
>> >>>>> >> I'm thinking that we probably want to test things like the halo
>> >>>>> >> finder, projections, and phase plots.  Would a medium (1536^3)
>> >>>>> >> halo
>> >>>>> >> profiling run do that?  Do we want to add on some global
>> >>>>> >> projections
>> >>>>> >> and phase plots as well?
>> >>>>> >>
>> >>>>> >> -Matt
>> >>>>> >> _______________________________________________
>> >>>>> >> yt-dev mailing list
>> >>>>> >> yt-dev at lists.spacepope.org
>> >>>>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > _______________________________________________
>> >>>>> > yt-dev mailing list
>> >>>>> > yt-dev at lists.spacepope.org
>> >>>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>> >
>> >>>>> _______________________________________________
>> >>>>> yt-dev mailing list
>> >>>>> yt-dev at lists.spacepope.org
>> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> yt-dev mailing list
>> >>>> yt-dev at lists.spacepope.org
>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Cameron Hummels
>> >>> Postdoctoral Researcher
>> >>> Steward Observatory
>> >>> University of Arizona
>> >>> http://chummels.org
>> >>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>