opendataology / webnotice-toolbox

webnotice-toolbox

Introduction

WebNotice-toolbox is a set of powerful web scraping tools designed to retrieve copyright notices from websites. This tool is especially useful for developers, content creators, and legal professionals who need to track and manage copyright information efficiently.

Features

• Efficient Scraping: Quickly scans and extracts copyright notices from a variety of websites.
• User-Friendly: Easy to set up and use, regardless of your technical background.
• Customizable: Options to specify which parts of a website to scan.
• Regular Updates: Continuously improved to handle new web technologies and patterns.

Contributing

Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the Apache License 2.0 License. See LICENSE for more information.

Contact

Project Link: https://github.com/OpenDataology/webnotice-toolbox

webnotice-toolbox's Issues

some common license url suffix

[" /chn/copyright.html " ,
" /terms.html " ,
" /html/cer.net/sm.htm " ,
" /rules/agreement " ,
" /agreement_wap.shtml?v=20171127 " ,
" /legal-notices " ,
" /m/ServicesAgreement/ " ,
" /agreement.html " ,
" /about/copyright.html " ,
" /about/statement.html " ,
" /marketing/legal_notice.html " ,
" /service.html " ,
" /help/article/2908 " ,
" /company/provisions " ,
" /upload/clientapp/agreement.htm " ,
" /about/copyright " ,
" /info/agreement/ " ,
" /forum-464900-thread-18281420697214773-1-1.html " ,
" /fuwu.htm " ,
" /inc/agreement/usercode.html " ,
" /help/copyright.php " ,
" /about/legal " ,
" /user_agreement/ " ,
" /group/about/flsm.shtml " ,
" /policies/user " ,
" /user/regagreement " ,
" /H5/help/user_privacy.asp " ,
" /copyright " ,
" /new-privacy-agreement/611279d8dbec4fdc?from=pc&bar=1 " ,
" /about/Legal%20Notices.html " ,
" /m/9c0092e0/kwpcuseragreement.html " ,
" /about/copyright/ " ,
" /help/3.html " ,
" /Register/NRegisterStruction.htm " ,
" /UserManage/login.aspx?agreement2 " ,
" /terms_of_service.html " ,
" /extend/copyright.html " ,
" /regagreement/ " ,
" /docs/policy " ,
" /com_statement.html " ,
" /site/about/law1.shtml " ,
" /about/ " ,
" /nfcf/banquan.html " ,
" /about/statemen.html " ,
" /aboutus/lawnotice.html " ,
" /web/common/protocol/protocol-2019-09-30.html " ,
" /t1008237/index.html?mscid=Fb1dVVx_&agreementType=A0001 " ,
" /aboutus/servicerule.html " ,
" /html/hybrid/other/user-agreement " ,
" /img/finance-service/user-service-agreement.pdf " ,
" /user_agreement.html " ,
" /serviceProtocol.html " ,
" /fe/protocol " ,
" /special/guanyunew/fuwuxieyi/index.shtml " ,
" /usercenter/web/userService.html " ,
" /company/4-5.html " ,
" /cloud/page/agreement.html " ,
" /about/protocol " ,
" /zh_cn/general/bqsm.html " ,
" /vip/right_declare/index.html " ,
" /faq/help_32.html " ,
" /m/9c0092e0/kwpcqlsm.html " ,
" /duty/copyright.html " ,
" /explain/service?type=service " ,
" /help/bqsm.html " ,
" /site-lawer.html " ,
" /pages/legal.htm " ,
" /user/register/protocol.html " ,
" /principle/ " ,
" /minisite/red/copyright/ " ,
" /copyright/ " ,
" /privacy.html " ,
" /help/agreement/ " ,
" /terms/ " ,
" /tos?utm_source=sf-footer " ,
" /tiaokuan.aspx " ,
" /intro/copyright.shtml " ,
" /obj/ies-hotsoon-draft/juejin/86857833-55f6-4d9e-9897-45cfe9a42be4.html " ,
" /tiaokuang.aspx " ,
" /home/legal " ,
" /pub/FAQ/Service.html#:~:text=%E7%94%A8%E6%88%B7%E5%90%8C%E6%84%8F%E4%BF%9D%E9%9A%9C%E5%92%8C%E7%BB%B4,%E7%9A%84%E6%8D%9F%E5%AE%B3%E8%B5%94%E5%81%BF%E8%B4%A3%E4%BB%BB%E3%80%82 " ,
" /zzsq/201811/t20181121_12114577.htm " ,
" /home/next?ch=useragreement " ,
" /zh-cn/statement.html " ,
" /html/duty.htm " ,
" /notice/banquan.htm " ,
" /about/4.html " ,
" /law.html " ,
" /about/notices.php " ,
" /help/law.aspx " ,
" /bo/service.php " ,
" /static/html/bbzdprotocol.html " ,
" /help/0032.html " ,
" /Index/law/ " ,
" /about/legal/ " ,
" /site/duty.html " ,
" /p/c44d171298ce " ,
" /info/copyrights.php " ,
" /about-copyright.html " ,
" /web/dsb_footer/bqsm/index.shtml " ,
" /shenming " ,
" /terms.php " ,
" /staticpage/agreement " ,
" /special/guanyunew/gongsijianjie/index.shtml?spm=C30123.PbAhUUiik0Hh.En9CqZ5t6KuD.5#wzsm " ,
" /s/agreement.html " ,
" /about-104.html " ,
" /help/fwtk/ " ,
" /register/terms.htm " ,
" /termsOfUse.aspx " ,
" /user/agreement " ,
" /html/about/rule.html " ,
" /codeindex.php?d=web&c=about&m=copyright " ,
" /portal/protocol?sourceid=220001 " ,
" /privacy4pcweb/v-e010f2d0/index.html " ,
" /bang/agreement.html " ,
" /daanxy.html " ,
" /help_lite/3a667def8c514aff8778dce22a691abd.html " ,
" /disclaimer/ " ,
" /about/laws " ,
" /about/agreement.html " ,
" /zhishichanquan.html " ,
" /member.php?mod=register.php " ,
" /about/falv.html?f=index&pvareaid=21256822772 " ,
" /common/webmodule/about/copyrights.aspx " ,
" /a/copyright.html " ,
" /about/tougao.aspx " ,
" /protocol/intellectualProperty.html " ,
" /help/helpcenter.html?type=9 " ,
" /Qikan/WebControl/ServiceAgreement?from=index " ,
" /Help/disclaimer.html " ,
" /about/fwtiaokuan.html " ,
" /jqw_legalNotice.html " ,
" /info/pact.jsp " ,
" /helpcenter/policy?id=44 " ,
" /yonghufuwu.htm " ,
" /regpage.aspx " ,
" /about/yibuService.html " ,
" /reg/agreement/ " ,
" /common/loginProtocol.html " ,
" /about/banquanyinsi.html " ,
" /details/page254 " ,
" /contract.shtml " ,
" /about/useragree " ,
" /item/801_0.html " ]