GithubHelp home page GithubHelp logo

simplejian / web_kg Goto Github PK

View Code? Open in Web Editor NEW

This project forked from lixiang0/web_kg

0.0 1.0 0.0 40.29 MB

爬取百度百科中文页面,抽取三元组信息,构建中文知识图谱

Python 100.00%

web_kg's Introduction

开源web知识图谱项目

环境

  • python 3.6
  • requests:网络请求
  • re:url正则匹配
  • bs4:网页解析
  • pickle:进度保存
  • threading:多线程
  • neo4j:知识图谱图数据库,安装可以参考链接
  • pip install neo4j-driver:neo4j python驱动

运行前指定几个路径:

spider/html_paser.py第38行为网页存储路径:

        path='/data/ruben/data/webpages/'#custom diectory for webpages

ie/extract-para.py第11行为网页存储路径:

        pages=glob.glob('/data/ruben/data/webpages/*')

ie/extract-table.py第37行为网页存储路径:

        pages=glob.glob('/data/ruben/data/webpages/*')

代码目錄

  • spider/ 抓取原始网页
  • ie/ 从网页中解析正文,从正文中抽取结构化信息
  • kg/ 抽取三元組,存入neo4j数据库

代码执行顺序:

python spider/spider_main.py
python ie/extract-para.py
python ie/extract-table.py
python kg/test_neo4j.py

web_kg's People

Contributors

lixiang0 avatar

Watchers

SimpleJian avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.