peeringdb / peeringdb

Server code for https://www.peeringdb.com/

PeeringDB

PeeringDB provides a scalable way to publish interconnection related information for tens of thousands of network operators. Over the years, more and more organisations have built integrations to streamline their interconnection processes.

The source code powering PeeringDB is publicly available under a BSD 2-Clause license.

We anticipate that work considered to be "low hanging fruit" could be taken on by volunteers, and we expect that open sourcing the platform makes it easier for third parties to participate in bidding for new features specified by the Product Committee.

Developers in the community may fork the code, create a container for development and testing, and submit pull requests.

peeringdb's Issues

integrate membership list

Site should allow for list of PDB members and authorized voters per org.

Drop IX Lans

see http://lists.peeringdb.com/pipermail/pdb-tech/2016-April/000104.html for discussion

Get by ASN only searches primary ASN

A network that has multiple ASNs isn't returned when requesting non-primary ASN.

For example, APNIC (id=3103) has the primary ASN 4608, but different ASNs at different IXs (for example ASN 18369 at Equinix-Ashburn (fac=1))

https://beta.peeringdb.com/api/asn/4608 returns APNIC's record, but https://beta.peeringdb.com/api/asn/18369 does not.

Searching on the web interface yields the same results (specifically, searchers for 18369 returns no results).

Shouldn't a get or search work for all ASNs for a given org?

Moved from peeringdb/docs/issues/35

Create versioning for static files

So we don't run into browser cache issues when javascript/css changes are released

Moved from peeringdb/docs/issues/99

Cant edit name of entities under organizations with sponsor status

Clicking the name textbox will take you to the sponsors page

add long/lat to schema and API

Fix out of office ticket spam

1. User A requests affiliation with record XYZ
2. An e-mail is sent to all admins (or even members???) of parent
   organisation of XYZ with from address [email protected]
3. If this e-mail hits an ooo a reply is sent to
   [email protected]

A decent header in the outgoing e-mails to the admin could prevent
backscatter.

use Return-Path: <> in e-mails

every no wand then support gets flooded by out of office e-mails, esp. when a user requests affiliation to an organisation with lots of admins.

Setting "Return-Path: <>" should do the trick according to RFC http://tools.ietf.org/html/rfc3834 or use Auto-Submitted header with value other than "no" to explicitly indicate automated response.

add notification e-mail address for IXP

Add a notification e-mail address to IXP records. Whenever a networks joins / leaves a notification is sent to this address

add long/lat script to populate data from address

Main is included in facility name

Our system had some trouble figuring out a peering name. We got the following section (from https://www.peeringdb.com/api/net?asn=8757&depth=2 but not really important).

    {
      "id": 29439,
      "ix_id": 18,
      "name": "LINX LON1: Main",
      "ixlan_id": 18,
      "notes": "",
      "speed": 100000,
      "asn": 8757,
      "ipaddr4": "195.66.225.49",
      "ipaddr6": "2001:7f8:4::2235:1",
      "is_rs_peer": true,
      "created": "2016-07-06T14:27:11Z",
      "updated": "2016-07-28T09:26:47Z",
      "status": "ok"
    }

Both the network engineer and I are surprised about the the inclusion about :Main in the name.
If you go to https://www.peeringdb.com/ix/18 the main part is not included, so it seems like this introduces a new name, which is not intuitive.

Pleaser consider putting the information about main facility in a seperate field.

organization management: sometimes unable to remove user permissions

Clicking the [x] button will remove the permission from the table, but permission will be back after page reload

Given 2 ASNs, tell me where they overlap

Use cases:

1. As an operator, I see significant traffic to an ASN and I want to peer with them. I don't know what IXes we are both at.

2. As an operator, I have public peering with an ASN, and I want to move to PNI because traffic volumes are high enough now. I want to see what facilities we have presence in besides the public ones.

3. As an operator, I don't have any common facilities with an ASN, but we're in the same metros. I want to see how difficult it would be to get from the facility/facilities I'm in in $metro to the ones they're in in $metro. This bleeds into #11.

Use case 3 is harder, and we need more data about facility <-> facility interconnection or equivalence to do properly. But 1 and 2 should be easy.

You could imagine showing this when logged in as an ASN. You browse to https://www.peeringdb.com/net/$X and you see where the net you're affiliated with overlaps with $X.

Something like this seems to work (but doesn't handle local_asn so could use some improvement...):

select distinct * from
(select 
    na.name isp_1, 
    nb.name isp_2, 
    fa.name, 
   fa.city city,
    fa.country country,
    'private' connection_type
from 
    net na, 
    net nb, 
    netfac fa, 
    netfac fb
where 
    na.id = fa.net_id 
    and nb.id = fb.net_id 
    and na.asn = $asn_1 
    and nb.asn = $asn_2 
    and fa.fac_id = fb.fac_id
union all
select 
    na.name isp_1, 
    nb.name isp_2, 
    ix.name, 
    ix.city city,
    ix.country country,
    'public' connection_type
from 
    net na, 
    net nb, 
    netixlan ila, 
    netixlan ilb,
    ix
where 
    na.id = ila.net_id 
    and nb.id = ilb.net_id 
    and na.asn = $asn_1 
    and nb.asn = $asn_2 
    and ila.ix_id = ilb.ix_id 
    and ix.id = ila.ix_id
) z

Remove regional from scope option

Will leave anything currently with regional as "Not Disclosed"

Add i18n

AdminCom has seen a significant uptake in support related work items from organisations residing in Portugese speaking countries. The language barrier sometimes makes things a bit difficult. I'd like to serve PeeringDB in Portugese in conjunction with the current English texts.

AdminCom can arrange for translations if a language abstraction engine of sorts is available.

Moved from peeringdb/docs/issues/90

Allow network's IXP IP to be null

This is to indicate that a connection to that IXP will happen and to give other participants a heads up

combine mailman and user db to create member list

Check-list for Membership:
- confirm pdb-gov mailing list membership
- confirm active PDB account for organization

Check-list for Registered Voters:
- confirm authorized representative for organization
- confirm only one membership per affiliated organizations

Membership count for the Bylaws quorum number. (presently 10%)

The list of Members can be automated if you can tie every subscription to
pdb-gov to a PDB 2.0 org account, keeping in mind that some orgs may want
to have multiple addresses subscribed, and also keeping in mind that some
subscription addresses may be associated with multiple orgs, and also
keeping in mind that some folks want to be on the digest version of
pdb-gov.

An org is a Member, if it has one or more addresses subscribed to the
pdb-gov or pdb-gov-digest lists.

Mailing list bounces/drops/unsubscriptions also need to be handled, such
that if an address is removed from the mailing list, it no longer
qualifies an org for membership.

atomic sync

Currently there are still instances where sync can malfunction due to way updates are sent out via the since endpoint on the API. While this is rare and avoidable by being vigilant about the data, having sync be atomic should fix this.

Support for multiple IRR records, identify source

The current database only has one field for a single IRR record, and many networks have multiple, for example separate for IPv4 and IPv6, or different IRR record for customers/peers/regions etc.

The source of the IRR record is not identified - is it in RIPE, RADB, ALTDB etc?

Is there any intention to allow multiple? What seems to happen now is that the field can contain multiple IRR records in a variety of interesting ways, and parsing this is of course a pain.

Moved from peeringdb/docs/issues/56

add address validator when entering new addresses

Add padding to the left of the sponsor banner

I think there should be padding between the name and banner.

Make it clear that some contact addresses are available by logging in

When the visibility of a contact is set to "Users", it is only displayed to logged-in users.

But when a visitor is not logged in, there is no indication at all that a contact address might be available by logging in. I frequently use peeringdb without being logged in, and I simply assumed that people did not provide a contact. Only recently did I discover that there are in fact contact addresses, but you need to be logged in to see them.

A solution to this would be to display the contact while hiding the actual email address / phone number. There would then be a tooltip (or whatever) explaining that you need to be logged in to actually see the address of the contact.

Having "create" permissions for "any facility" does not enable UI for facility creation

When an organization member is given "create" permissions to "any facility" the ui to create a facility does not show up for them on the organization administration page.

When organization has users to approve auto-focus on Users tab when loading the page

Quality of life improvement for organization admins

No feedback when adding a facility or exchange to a network while the list is filtered.

What I do is filter on for instance 'amsterdam' and then adding doesn't add it to the list

Admin users are unable to add facilities / IXP

http://ubersmith.peeringdb.com/admin/supportmgr/ticket_view.php?ticket=14000

http://ubersmith.peeringdb.com/admin/supportmgr/ticket_view.php?ticket=14016

Include latitude/longitude on facilities

a) You can put 'em on a map easily.
b) You can show how close facilities are (or if they're in the same building as described.
c) You can use the lat / long to cluster facilities and fix missing CLII codes.

make websearch smarter

if you search for ntt global you do not get results, even though a NTT Communications (Global) network exists, it seems there is no tokenizing & indexing on the searchable content. Maybe we should pump records into elasticsearch to have a more 'google-like' search experience?

Moved from peeringdb/docs/issues/77

add user agent and version check to server

Will be useful for giving informative error messages as features are added.

Improve information text at "Manage" section for "Add IXP" and "Add Exchange"

A lot of users don't get it that this section is only to add own facilities, networks and IXPs. So, please make clear that this section is only there to add records you own.

For Exchanges please also add the information that the Exchange will only show up in the search list for adding IXP if the organisations has provides ixlan information as well

pdb sync issues

Operations to perform:
  Synchronize unmigrated apps: django_peeringdb
  Apply all migrations: admin, contenttypes, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  No migrations to apply.
Starting new HTTPS connection (1): www.peeringdb.com
"GET /api/org?since=1469134856 HTTP/1.1" 200 None
org last update 1469134856 0 changed
data to be processed 0
Starting new HTTPS connection (1): www.peeringdb.com
"GET /api/fac?since=1469134856 HTTP/1.1" 200 None
fac last update 1469134856 0 changed
data to be processed 0
Starting new HTTPS connection (1): www.peeringdb.com
"GET /api/net?since=1469131373 HTTP/1.1" 200 None
net last update 1469131373 4 changed
data to be processed 4
Traceback (most recent call last):
  File "$PATH/pdbvenv/bin/peeringdb", line 9, in <module>
    load_entry_point('peeringdb==0.4.4.1', 'console_scripts', 'peeringdb')()
  File "$PATH/pdbvenv/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "$PATH/pdbvenv/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/peeringdb/cli.py", line 167, in sync
    db.sync()
  File "$PATH/pdbvenv/lib/python2.7/site-packages/peeringdb/localdb.py", line 140, in sync
    call_command('pdb_sync', interactive=False)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 85, in handle
    self.sync(tables)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 92, in sync
    self.update_db(cls, self.get_objs(cls))
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 127, in update_db
    sync.sync_obj(cls, row)
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django_peeringdb/sync.py", line 25, in sync_obj
    obj.full_clean()
  File "$PATH/pdbvenv/lib/python2.7/site-packages/django/db/models/base.py", line 1171, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'name': [u'Network with this Name already exists.']}

Make peeringdb-py client use local cache for commands

Currently, queries from the client still goto the server, should be changed to allow for using a local cache.

Allow for username retrieval/recovery

Hiya,

It would be great if a username recovery functionality would exist alongside password recovery. Users could enter their email address and have their username sent to them, just like in password recovery.

Contact fields removed extra spaces, new line chars, and html formating

Minor spacing issue in HTML code for all contact fields. This is causing issues when copying and pasting email address from peeringdb. You end up copying the spaces and newline chars along with the email address. I am using chrome. This happens on all peeringdb records not just this one entry.

The html code for contact names, phone, and email has the following. Below is an example which is an email address. From: view-source:https://www.peeringdb.com/net/917 line 891. There are extra spaces and new lines.

             <div class="email"

                  data-edit-value="[email protected]"
                  data-edit-name="email">
                [email protected]
             </div>

When it should be this so copying an pasting doesn't pickup the new line chars and spaces.

             <div class="email" data-edit-value="[email protected]" data-edit-name="email">[email protected]</div>

Support for Multiple IP Addresses on an IX

Is it possible to add support for adding multiple IP address for a single IX?

For example an ISP could have multiple IP addresses on a single interface to an IX but you can only list one IX Address.

If you need to add another you need to create a new 'line' in your PDB record and associate a capacity with it, therefore you could be misrepresenting your capacity.

Therefore I would like the ability to add multiple addresses to an IX using the same capacity.

recaptcha doesn't work in China

Which blocks the signup process. Should either proxy for Chinese addresses, or find a different captcha solution

UI for managing authorized voters

maybe there should be a "Governance" tab in the "Manage" section. Under
there it could say:

• Bylaws (a href to
  http://docs.peeringdb.com/gov/legaldocs/2015-12-08_PeeringDB_Bylaws.pdf)
  define Membership:

  "A corporation, limited liability company, partnership or other
  legal business entity may be a Member of the Corporation. Membership
  is determined by having both an active PeeringDB.com account and an
  individual representative or role subscription to the PeeringDB
  Governance mailing list:
  http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-gov"

and then there could be a way to add/remove subscriber email addresses,
along with a checkbox for selecting "Digest Subscription".

As far as displaying the Member list, maybe it would be linked to from the
footer "Global System Statistics", via a line that says "### Members".
The list should only show organization name. Definitely not email
addresses.

Phone number format

every now and then I come across "suspicious" phone numbers. Are you
enforcing E.123 international notation for phone numbers? And how do we clean up
all the "garbage"?

Firefox unable to scroll auto-complete dropdowns

There is an issue where trying to scroll auto-complete results - like facilities and exchanges - in firefox will instead close the auto-complete result.

IXP and Facility summary

It would be nice to have a summary for IXPs. For example the number of connected peers and how many have open/selective/restrictive peering policy.

Moved from peeringdb/docs/issues/107

Check network's IXP IP address against IXP's IP range

Make sure that $networks_IP is within the IXP's IP range (both IPv4 and IPv6)

When adding facilities, auto complete should exclude already added ones

Add operation to API to look for covering prefixes given an IP

While testing the REST API to lookup if a given IP address belongs to an IX in the PeeringDB database, I came up with the less than ideal solution:

For address 206.223.118.117 search as

https://beta.peeringdb.com/api/ixpfx?prefix__startswith=206.223.118

Will return a prefix and a ixlan_id that can be used to query up to the organization.

But if the prefix is not aligned with the octet (v4) or nibble (v6), the solution doesn't work.
Have you considered an special lookup for find matching network? The
simplest option is "find a prefix that covers this address", although
the generic option is "find a prefix that covers this prefix", and then addresses can be search as 'x.x.x.x/32' or 'dead::beef/128'

If the backend code is available somewhere, I'll be happy to provide a patch.

Moved from peeringdb/docs/issues/61

Whois: an error occurred

Reported by Peter Hessler on BSD client, but the same error happens on Linux too:

whois -h whois.peeringdb.com as20940

blah blah

Interxion Madrid (MAD1, MAD2) 20940 Madrid ES
Interxion Frankfurt 2 20940 Frankfurt DE
KDDI Otemachi (Telehouse Tokyo Otemachi) 20940 Tokyo JP
Interxion Copenhagen 20940 Copenhagen DK
an error occurred

Moved from peeringdb/docs#104

Model campus datacenter facilities

Some datacenters are in campus buildings, named Facility DC1 to Facility DCn. From outside it may look like different DCs, in reality it's just one. See Ticket #8549 in Ubersmith.

Idea is to have one virtual DC and keep the structure. This might also model a DC in a DC perhaps

the server rejected your data The fields network, facility, local_asn must make a unique set

When trying to add facility "Global Switch (London 2)" to the set of fcilities of ASN 6774 this error message pops up.

add better max prefix support

See discussion at peeringdb/docs/issues/38

Long requests gives "502 Bad Gateway"

When I do

curl -s -k -X GET https://www.peeringdb.com/api/net\?id__in=3881,7881,6723,6724,6725,3884,6726,3570,6494,6453,7887,3220,6186,3222,6106,3265,3225,781,702,4880,785,8882,86,4842,433,475,709,120,1964,121,163,1925,4889,1651,479,4260,124,126,1,1613,1614,2,1656,4,4224,5,1385,1304,8,2961,5532,2927,5222,5263,2382,2698,2616,5539,2699,2344,5229,2033,2078,2037,3921,3693,3615,6266,3618,3386,3308,821,590,823,4960,4961,4964,596,283,599,203,1772,286,4697,4341,8383,1777,8303,208,4070,1422,1423,1192,4034,1111,4308,1467,1193,4035,1112,1427,4036,1469,4079,1197,1157,1159,5964,9650,5927,2462,5071,2464,5388,2152,5309,5077,6652,6611,7771,3736,6386,3465,3152,3195,3155,3198,981,983,670,986,947,906,633,360,635,678,4734,4735,1812,1855,327,4190,1815,1501,8153,4196,1506,1233,1234,4199,1236,1279,9777,2895,9464,2501,2858,2819,5195,2273,5199,2235,2277,2236,6732,6460,3856,6464,6424,3502,3276,7236,7238,91,4890,4850,13,14,4894,170,8581,17,99,4814,8541,8815,4500,487,1660,1977,4231,8506,1624,1625,8549,4549,1354,1355,1357,1317,1319,5893,9812,2971,2625,9233,5233,2310,2313,2084,2006,6852,3932,3974,3938,3623,7350,6589,3312,3668,3080,3629,3397,3042,7002,7318,7319,7048,873,521,837,291,250,8977,4977,4937,8663,4622,254,4352,258,217,8041,1748,1476,8359,1477,1163,1479,1124,1125,9622,9938,2702,9311,5626,2704,2430,2431,1081,5042,2435,2477,5004,1086,1045,1004,1046,1006,9089,1007,6970,6939,6350,6312,3432,3707,6043,3436,3167,950,914,997,601,684,4740,4782,648,8745,4745,4704,4430,1864,10147,10106,1280,1555,1240,4122,1282,1201,1598,1517,1599,1204,9471,5432,2591,2592,5433,2595,2596,2281,2241,9124,2283,2558,2202,2519,6782,3554,3241,7285,3559,3208,721,763,21,450,725,23,1981,1900,27,457,29,1903,184,1904,499,458,1946,186,104,1949,1675,1635,149,1677,1363,1367,9590,2943,2986,2630,5595,2905,2988,2906,5282,2989,5283,5516,9284,2366,2325,5208,2327,2095,2096,2098,6860,3900,6591,3671,6596,6243,3320,3324,7327,3327,3055,3014,3016,3058,3017,572,8901,261,576,577,4631,262,4987,263,222,1792,1751,1752,539,1753,1480,268,269,1483,4092,1799,1443,1170,1486,8013,4018,8019,1138,9940,5320,5321,2755,5010,2404,2173,5014,2132,1014,2174,1098,6362,7715,3715,3716,7758,7487,3131,965,694,380,4710,341,301,617,8795,1872,4714,1833,346,347,4402,1836,308,309,8406,1565,4449,4176,1569,1258,5483,2521,9171,5171,2250,9448,2214,2218,3878,6132,7292,3567,7259,731,72,733,31,75,779,4874,423,8560,78,150,110,1955,1914,1956,1682,1915,1641,196,1683,155,115,4211,1918,1644,198,116,1371,4294,8295,1372,118,4257,2995,2600,2957,5210,2648,5217,2062,5259,2024,2028,3992,7954,3332,3647,6215,3060,3608,7063,3024,890,891,893,854,541,8993,4911,585,270,505,4641,4916,273,1721,508,10003,1764,236,278,1724,1766,1451,279,1726,1452,1412,1728,4338,1498,1459,1418,1186,1189,1108,5992,5994,9640,5641,5918,2491,9607,2494,2727,1062,2728,2180,2181,5022,2496,9379,2456,2100,9065,5025,1066,5068,6993,6995,6680,3725,3411,6061,3497,3142,3101,3418,972,3149,621,662,979,311,4722,316,1803,1804,358,4727,319,8497,8182,1848,4498,4418,1221,4103,1538,1264,1223,4105,1265,1268,1227,2800,2885,5493,2803,2534,5499,2577,2536,5188

I get "502 Bad Gateway"

improve approval notifications

All the admin level users within an organisation get the User XX YY wishes to be affiliated to your Organization emails, but it would be useful if there is a follow-up email to the same group of email addresses that says Admin ZZ approved or denied User XX YY to be affiliated with organisation AA. This way people know that the request has been handled and who handled it.

Moved from peeringdb/docs/issues/103

Obfuscate email addresses

It seems that currently, publicly visible email addresses are completely unobfuscated, which is very nice for spammers collection scripts.

It would be nice to at least attempt to obfuscate them (I don't know the state of the art, maybe javascript?). Without that, I know I will never publish the contact information for my ISP as public.