kipac / statisticalmethods Goto Github PK

For all to fill in. Lets make it a Glossary.md file in doc. Thanks @rhw!

@drphilmarshall, I added some notes on importance sampling and PMC, tentatively for inclusion in session 4, but could be elsewhere if we think it makes sense.

Which is to say: what exercises do we want to do? And then, what's a good dataset to use? Suggestion so far: historical cepheids for H0! Assigning this to @elikrause to do with @beckermr but TBH we should all talk about this.

Copy FirstLook, do some masking and summarizing.

astropy
ds9

Might allow us to bypass quite a lot of crappy bookkeeping.

• Edit XrayImage notebook to replace pyfits with astropy
• Add astropy to the required packages in the Getting Started doc.

A plain markdown file awaits us, here!

Was updated over the weekend. Now supports options for much nicer looking graphics (see examples in @drphilmarshall notebooks(s)).

pip install --upgrade triangle_plot

Scripting the random selection of students to present homework in class means the easy generation of a machine-readable file: google spreadsheets works well for this, and we can scrape out the "submissions" via the csv output. The link to the form

To do:

• Make a test form and try it out
• Implement a simple command-line script to scrape the csv, and draw random presenters for each question.

Introduce correlation function as the ultimate summary statistic. Compute it for a small sample of SDSS galaxies, get something very noisy. Invite comment.

The idea is to set up some of the cosmology stuff in Week 3, and also produce a set of summary statistics that look like the Cepheid dataset. Then, we can talk about the meaning and interpretation of the error bars...

Download, plots of data with and without errors. While doing this, think about/investigate possibility of correlation function summary, which would lead into the 3rd datatype (shown in Cepheids)

If students don't have their own laptops will they have access to git and ipynb through Stanford or physics computers?

This means that the a1835_xmm folder also needs to be checked in (and so not created by teh FirstLook notebook. At the moment we make it with !mkdir -p a1835_xmm - this will be redundant, but not dangerous, when you have checked in. However, the .gitignore file will need editing, to not ignore a1835_xmm but instead ignore *.FTZ Thanks!

Based on notes in doc. Will need to start with recap of some sort. Think about/plan active exercises before getting into any content.

Link to the list of actual packages

FYI, I stumbled upon a book here. At first glance, it looks very thorough, although only a few chapters are online.

We can write this here, on Monday.

Let's code this together based on your homework so far, @abmantz

We need a notebook to enable exploration of a Metropolis sampler in the context of the XrayImage.
Requirements:

• Performs MCMC sampling of simple model parameters given image
• Enables exercises on speed-ups
• Enables easy switching in of emcee (either in class of for homework)
• Enables importance sampling demos: 1) re-using samples with new prior, 2) adding in new constraint (exercise)

As emailed, or on the DESC confluence

• @elikrause
• @abmantz
• @beckermr
• @rhw
• @drphilmarshall

Using matplotlib, and some suitable color table.

For future reference, the matplotlib gallery is incredibly useful.

Including top level notebook, 0.ThePlan.ipynb (which just contains the list of lectures, for easy navigation). Each lesson notebook should contain at least: a heading, the goals for that lesson, and related reading.

Do we have everything that needs saying at the start of class in place at https://github.com/drphilmarshall/StatisticalMethods/blob/master/notes/About.ipynb ? At what point do you want to hand over to us, @rhw ?

Markdown file(s) with:

• How to get the course materials (fork the repo)
• How to install python and work with notebooks
• Which books we recommend and where/how to get them

Probably one file, with links to sub-files, is a good idea. And all this can live in doc

One folder per example dataset, and make stubs of notebooks based on existing material.

• examples/StraightLine (see #8 and #11 )
• examples/XrayImage (see #9
• examples/SDSScatalog (see #10)

Becky pointed out the existence of this to me.
https://pypi.python.org/pypi/astroquery
It interfaces with lots of databases, including SDSS (mostly but not all optical, unfortunately not any of the X-ray dbs). Might be useful for the SDSS module.

We'll need these. I'll make a start and leave this open.

@drphilmarshall Please send me the link to Gaussian identities that I should discuss in lesson 3

• Remove "this page is maintained" etc to try and improve sidebar
• Remove download options (except for View on GitHub)
• Add a note that to see the course materials, you need to see the notebooks on GitHub or preferably at home in a notebook

@drphilmarshall and I are sketching out the comparison between Bayes factors and p-values for a simple detection problem on his whiteboard, and we're going to code it up for Lesson 6 (Model Evaluation). Comments welcome!

Track down Bloom'sstarter, check license, then import, break up and check in.

Include high level course overview as well as this particular lesson. May need some logistic stuff before getting into the Goals for today.

i.e. move code from bash script to notebook directly.

@abmantz see this other notebook for an example of how to do this, including a test to see if you already have the data.

Change code for randoms from flat to curved sky

Based on survey responses!

Hi all!

Homework assignments will be made available to git pull from the 2015 private homework repo. We'll grant read (i.e. fork) access to each of you taking the course, but only once we recognize you (and have matched you to the Stanford sign-up sheet). To be able to do this, we'll need you to introduce yourself on this issue thread (by writing a comment), and also to able to see your real name and your photo on your profile, please! Why not tell us all a bit about yourself while you're at it. What are you most interested in, in astronomy?

Identify "key message" bullet points for these lectures to build activities around. Share these somewhere on the github.

That none but we teachers shall see! First task once we have this up: check in @abmantz' s XrayImage MCMC stuff ( #27 )

A couple of them have remarked to me that they haven't heard anything official even though they know that there's some beforehand setup needed on their laptops. Might be a good idea to point them to the website soon.

Note GPL licence! That sets a pattern...

Simplest possible generative model, size of galaxies at fixed mass.

• Simulate draw from mock population
• Simulate observational uncertainty
• Look at effect on size variance in "sample"
• Discuss "one galaxy" interpretation
• Introduce sampling distribution, conditional PDF
• First PGM.