deercoder / hadoop-practice Goto Github PK

For hadoop's reduce(), here I just use empty(which means number zero) to output one collection. For example, if there is a city that satisfy the requirement, I just output (city, 0), as reduce() or map() both needs to operate as a tuple. But as a solution it's actually not so good.

But I think it's not elegant, there must be method to eliminate the zero so that there is only the city list for ex1.

Problem Set 1: output the cities where the populations are more than 300,000.

Problem: I will output the city name as well as an 0 behind it.

These two projects are not elegant, many of codes are poorly written and contains some format error, I need to rewrite them in good way...

In ex1, I output the city name, but it seems that the format of hadoop don't support unicode, so my solution will display abnormally, this is a bug.

And I think hadoop should be good at dealing this issue, since many data contains regional code and characters in different languages.

Leave a mark here and will solve it soon.

ex3 doesn't actually output all the countries that is English, it outputs ALL the countries. We have to add other judge conditions.

as title shows, wrong coding for the problem set, don't meet the requirements.

It's quite strange that I can execute the map() function, no error is there, and there is also something output there, but it seems that our map() function is actually not called, since at the very first beginning, log is not output.

Then why this bug happens? And why there is no error?