craw_stock's Introduction

起因

因一个审计专业同学需要对上市公司的利润表中数据进行分析，因此帮忙写了爬虫然后用K-means简单分析了一下，找出可疑公司

爬虫部分

爬虫部分一共爬取了两个网站的数据，因为第一个网站的数据，爬取的第一个网站是同花顺，爬虫代码在selenium_com.py文件中，用mongodb数据库存储

第二个网站是东方财富网，代码在craw_dfcfw.py中，保存方式为CSV文件，保存在dataset文件夹下

详细过程有空补充

聚类部分

因为审计专业的要求和计算机不太一样，他们这个方法的要求是先通过K-means算法进行聚类，然后把聚类结果里面的小簇（就是只有几个数据的那种）中的数据给删除掉，因为这种数据说明该公司的财务数据存在疑点，而他的论文正是要找出这些可疑的公司和数据，然后将这些可疑点给删除，再次进行聚类（论文中称之为迭代式聚类），反复多次，直到没有小簇出现，以此来达到筛选出存在审计疑点的公司

细节待补充

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.

Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

TensorFlow

An Open Source Machine Learning Framework for Everyone

Django

The Web framework for perfectionists with deadlines.

Laravel

A PHP framework for web artisans

D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

web

Some thing interesting about web. New door for the world.

server

A server is a program made to process requests and deliver data to clients.

Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

Visualization

Some thing interesting about visualization, use data art

Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.

Microsoft

Open source projects and samples from Microsoft.

Google

Google ❤️ Open Source for everyone.

Alibaba

Alibaba Open Source for everyone

D3

Data-Driven Documents codes.

Tencent

China tencent open source team.

lovejun277 / craw_stock Goto Github PK