Comments (4)
Verify that the interpolation step is handled correctly. Per the EPI methodology:
For these reasons, we enforce some smoothness in the wage data to ensure that exactly 10 percent of our sample falls at or under the 10th percentile wage (and 20 percent falls under the 20th percentile and so on). The smoothing technique involves creating a categorical hourly wage distribution, in which the categories are 25-cent intervals (also referred to as 25-cent bins). We then find the bins on either side of each decile and perform a weighted, linear interpolation to locate the wage cutoffs for each of the particular deciles. The weights for the interpolation are derived from differences in the cumulative percentages on either side of the decile. For example, suppose that the 48th percentile of the wage distribution of workers by wage level is in the $9.51–$9.75 wage bin, and the 51st percentile is in the next higher bin, $9.76–$10.00. The weight for the interpolation (in this case, the median, or 50th percentile) is (50–48)/ (51–48), or two-thirds. The interpolated median equals this weight, times the width of the bin ($.25), plus the upper bound of the previous bin ($9.75); $9.92 in this example.
from econ_data.
By shrinking the bin size, the results look much better. Still, I have not verified that the methodology matches that used by Census/BLS/EPI. Closing the issue.
from econ_data.
There are still large jumps in the data. They are disguised by the moving average but much larger than I would expect. Reopening the issue to investigate.
from econ_data.
I've been able to match BLS data fairly precisely. I'm not sure what the problem was in the cases, but if it happens again I will re-open this and add an actual example.
from econ_data.
Related Issues (20)
- KeyError: PRSJMJ in apr20pub.dat file HOT 3
- Jan99dd2.asc created manually (?) HOT 2
- bd CPS: issue from use of 2000-02 revised weights HOT 2
- bd CPS: HHWGT incorrect in early 1995 HOT 2
- bd CPS: remove grapher and similar HOT 1
- bd CPS: benchmark of family count fails HOT 1
- bd CPS: add headship variable HOT 1
- bd CPS: New variable DISCTYPE HOT 1
- bd CPS: micro: Living with Family / Headship
- bd CPS: fix variable METSTA
- bd CPS: New variables - Disability type
- bd CPS: extend variable SCHOOL
- bd CPS: new variable FAMINC
- micro: Occupational separations
- bd CPS: Consider CPI variable
- bd CPS: New variable PTREASON HOT 1
- bd CPS: benchmark percentile calculation
- bd CPS: uhoursi shouldn't use actual hours as is
- micro: CBSA split by state
- No field of name NWSSWGT
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from econ_data.