GithubHelp home page GithubHelp logo

wubinzzu / neurec Goto Github PK

View Code? Open in Web Editor NEW
1.0K 42.0 215.0 18 MB

Next RecSys Library

Python 92.80% C++ 1.90% Jupyter Notebook 3.81% PowerShell 0.12% Shell 0.12% Cython 1.26%
recommender-systems neural-networks collaborative-filtering deep-learning tensorflow variational-autoencoder autoencoder convolutional-neural-networks adversarial-learning sequential

neurec's Introduction

NeuRec

An open source neural recommender library

Main Contributors: Bin Wu, Zhongchuan Sun, Xiangnan He, Xiang Wang, & Jonathan Staniforth

NeuRec is a comprehensive and flexible Python library for recommender systems that includes a large range of state-of-the-art neural recommender models. This library aims to solve general, social and sequential (i.e. next-item) recommendation tasks, using the Tensorflow library to provide 33 models out of the box. NeuRec is open source and available under the MIT license.

Features

  • Cross platform - run on any operating system with the available Docker images;
  • State-of-the-art - 33 neural recommender models available out of the box;
  • Flexible configuration - easily change the configuration settings to your exact requirements;
  • Easy expansion - quickly include models or datasets into NeuRec;
  • Fast execution - naturally support GPU, with a mutli-thread evaluator;
  • Detailed documentation - extensive documentation available as Jupyter notebooks in tutorial.ipynb.

Architecture

The architecture of NeuRec is shown in the diagram below:

Quick Start

Firstly, download this repository and unpack the downloaded source to a suitable location.

Secondly, go to './NeuRec' and compline the evaluator of cpp implementation with the following command line:

python setup.py build_ext --inplace

If the compilation is successful, the evaluator of cpp implementation will be called automatically. Otherwise, the evaluator of python implementation will be called.

Note that the cpp implementation is much faster than python.

Thirdly, specify dataset and recommender in configuration file NeuRec.properties.

Finally, run main.py in IDE or with command line:

python main.py

Tutorial

The short introduction to NeuRec is described in tutorial.ipynb

Models

The list of available models in NeuRec, along with their paper citations, are shown below:

General Recommender Paper
GMF, MLP, NeuMF Xiangnan He et al., Neural Collaborative Filtering , WWW 2017.
BPRMF Steffen Rendle et al., BPR: Bayesian Personalized Ranking from Implicit Feedback. UAI 2009.
FISM Santosh Kabbur et al., FISM: Factored Item Similarity Models for Top-N Recommender Systems. KDD 2013.
NAIS Xiangnan He et al., NAIS: Neural Attentive Item Similarity Model for Recommendation . TKDE2018.
DeepICF Feng Xue et al., Deep Item-based Collaborative Filtering for Top-N Recommendation. TOIS 2019.
ConvNCF Xiangnan He et al., Outer Product-based Neural Collaborative Filtering . IJCAI 2018.
DMF Hong-Jian Xue et al., Deep Matrix Factorization Models for Recommender Systems. IJCAI 2017.
CDAE, DAE Yao Wu et al., Collaborative denoising auto-encoders for top-n recommender systems. WSDM 2016.
MultiDAE, MultiVAE Dawen Liang, et al., Variational autoencoders for collaborative filtering. WWW 2018.
JCA Ziwei Zhu, et al., Improving Top-K Recommendation via Joint Collaborative Autoencoders. WWW 2019.
IRGAN Jun Wang, et al., IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models. SIGIR 2017.
CFGAN Dong-Kyu Chae, et al., CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks. CIKM 2018.
APR Xiangnan He, et al., Adversarial Personalized Ranking for Recommendation. SIGIR 2018.
SpectralCF Lei Zheng, et al., Spectral Collaborative Filtering. RecSys 2018.
NGCF Xiang Wang, et al., Neural Graph Collaborative Filtering. SIGIR 2019.
WRMF Yifan Hu, et al., Collaborative Filtering for Implicit Feedback Datasets. ICDM 2008.
LightGCN Xiangnan He, et al., LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR 2020.
Social Recommender Paper
SBPR Tong Zhao et al., Leveraging Social Connections to Improve Personalized Ranking for Collaborative Filtering. CIKM 2014.
DiffNet Le Wu et al., A Neural Influence Diffusion Model for Social Recommendation, SIGIR 2019.
Sequential Recommender Paper
FPMC, FPMCplus Steffen Rendle et al., Factorizing Personalized Markov Chains for Next-Basket Recommendation, WWW 2010.
HRM Pengfei Wang et al., Learning Hierarchical Representation Model for NextBasket Recommendation, SIGIR 2015.
NPE ThaiBinh Nguyen et al., NPE: Neural Personalized Embedding for Collaborative Filtering, ijcai 2018.
TransRec Ruining He et al., Translation-based Recommendation, SIGIR 2015.
Caser Jiaxi Tang et al., Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding, WSDM 2018.
Fossil Ruining He et al., Fusing similarity models with Markov chains for sparse sequential recommendation, ICDM 2016.
GRU4Rec Balázs Hidasi et al., Session-based Recommendations with Recurrent Neural Networks, ICLR 2016.
GRU4RecPlus Balázs Hidasi et al., Recurrent Neural Networks with Top-k Gains for Session-based Recommendations, CIKM 2018.
SASRec Wangcheng Kang et al., Self-Attentive Sequential Recommendation, ICDM 2018.
SRGNN Shu Wu et al., Session-Based Recommendation with Graph Neural Networks, AAAI 2019.

Acknowledgements

The development of NeuRec is supported by the National Natural Science Foundation of China under Grant No. 61772475. This project is also supported by the National Research Foundation, Prime Minister’s Office, Singapore under its IRC@Singapore Funding Initiative.

next

neurec's People

Contributors

davidchenlondon avatar jaireyu avatar shawwang1995 avatar wubinzzu avatar zhongchuansun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

neurec's Issues

Question about adding negative instances to the training set

Thank you very much for your efforts in creating this great library. I have a question regarding the generation of the training set as follows. According to my understating, a number of negative instances is added to the training set, i.e., _get_pointwise_all_data(dataset, num_negatives) function in util/DataGenerator.py
However, the goal of the item recommendation problem is to recommend N items to a user given the set of items the user has interacted with. Thus, adding negative instances to the training set seems violate the assumption of the item recommendation problem.

Could you please explain the rational behind adding negative instances to the training set and why is it reasonable?

Duc Nguyen

The inputs of Dataset() might be wrong in main.py

In the main.py, line 45 (

NeuRec/main.py

Line 45 in cde6833

dataset = Dataset(data_input_path,dataset_name,splitter,separator,threshold,evaluate_neg,splitterRatio)
), it is
dataset = Dataset(data_input_path,dataset_name,splitter,separator,threshold,evaluate_neg,splitterRatio) .
However, in the init method of Dataset, it says
def __init__(self,path,dataset_name, data_format, splitter,separator,threshold,evaluate_neg,splitterRatio=[0.8,0.2])

Possibly, an input argument dataset_format is missing in main.py, line 45. It should be

dataset = Dataset(data_input_path, dataset_name, dataset_format, splitter, separator, threshold, evaluate_neg, splitterRatio)

DeepICF & NAIS issue with loss

DeepICF and NAIS models generates the following error on line 215 for DeepICF:

loss, _ = self.sess.run((self.loss, self.optimizer), feed_dict=feed_dict)
and line 216 for NAIS:
loss, _ = self.sess.run((self.loss, self.optimizer), feed_dict=feed_dict)

ValueError: setting an array element with a sequence.

Screenshot_2019-11-06 Untitled

The properties for DeepICF is set to the following:

[DEFAULT]
data.input.path=neurec
data.input.dataset=ml-100k.rating
data.column.format=UIRT
data.convert.separator='\t'
data.convert.binarize.threshold=0
data.splitter=loo
data.splitterratio=[0.8,0.2]|float
rec.number.thread=20
rec.evaluate.neg=0
recommender=deepicf
topk=[10, 20, 30, 40]|int
gpu_id=0

pretrain_file=None

#Interval of evaluation
verbose=1

#adagrad,rmsprop,adam
learner=adam

#user: generate batches by user, fixed:batch_size: generate batches by batch size
batch_size=256

#Number of epochs.
epochs=300
#Size of each layer
layers=[64,32,16]|int

weight_size=16

embedding_size=16

#Index of coefficient of embedding vector
data_alpha=0

#Regularization for user and item embeddings.[1e-7,1e-7,0]
regs=[0.000001,0.000001,0.00001]|float
#L_2 regularization on each layer weights.
regw=[10,10]|int
#Index of coefficient of embedding vector
alpha=0

#Index of coefficient of sum of exp(A)
beta=0.5

#Number of negative instances to pair with a positive instance.
num_neg=4

learning_rate=0.001

#Activation for ReLU, sigmoid, tanh.
activation=Relu

#0 for deepICF_prod, 1 for deepicf_concat
algorithm=1

#Whether to perform batch norm (0 or 1)
batch_norm=1

#tnormal: truncated_normal_initializer, uniform: random_uniform_initializer,
#normal: random_normal_initializer, xavier_normal, xavier_uniform, 
#he_normal, he_uniform. Defualt: tnormal
embed_init_method=tnormal
weight_init_method=he_normal
bias_init_method=he_normal
stddev=0.01

The properties for NAIS is set to the following:

[DEFAULT]
data.input.path=neurec
data.input.dataset=ml-100k.rating
data.column.format=UIRT
data.convert.separator='\t'
data.convert.binarize.threshold=0
data.splitter=loo
data.splitterratio=[0.8,0.2]|float
rec.number.thread=20
rec.evaluate.neg=0
recommender=nais
topk=[10, 20, 30, 40]|int
gpu_id=0

pretrain=1

#Interval of evaluation
verbose=1

#adagrad,rmsprop,adam
learner=adam
#user: generate batches by user, fixed:batch_size: generate batches by batch size
batch_size=256

#Number of epochs.
epochs=100

weight_size=16

embedding_size=16

#Index of coefficient of embedding vector
data_alpha=0

#Regularization for user and item embeddings.
regs=[1e-7,1e-7,1e-5]|float

#Index of coefficient of embedding vector
alpha=0

#Index of coefficient of sum of exp(A)
beta=0.5

#Number of negative instances to pair with a positive instance.
num_neg=4

learning_rate=0.001

#Activation for ReLU, sigmoid, tanh.
activation=Relu

#0 for NAIS_prod, 1 for NAIS_concat
algorithm=0

is_pairwise=false
#pairwise: BPR(BPRFPMC),hinge,square
#pointwise: cross_entropy,square
loss_function=cross_entropy
#tnormal: truncated_normal_initializer, uniform: random_uniform_initializer,
#normal: random_normal_initializer, xavier_normal, xavier_uniform, 
#he_normal, he_uniform. Defualt: tnormal
embed_init_method=tnormal
weight_init_method=he_normal
stddev=0.01
pretrain_file=None

Tuning models

Hi,
Thank you for this great framework.
Kindly, do NeuRec support hyperparameters tuning in general_recommender?
Are there any examples?

Regards

Project dependencies may have API risk issues

Hi, In NeuRec, inappropriate dependency versioning constraints can cause risks.

Below are the dependencies and version constraints that the project is using

numpy==1.16.4
scipy==1.3.1
pandas==0.17
tensorflow==1.12.3

The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict.
The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.

After further analysis, in this project,
The version constraint of dependency numpy can be changed to >=1.8.0,<=1.23.0rc3.
The version constraint of dependency scipy can be changed to >=0.12.0,<=1.7.3.
The version constraint of dependency pandas can be changed to >=1.0.0,<=1.2.5.

The above modification suggestions can reduce the dependency conflicts as much as possible,
and introduce the latest version as much as possible without calling Error in the projects.

The invocation of the current project includes all the following methods.

The calling methods from the numpy
distutils.core.Extension
numpy.linalg.eig
distutils.core.setup
The calling methods from the scipy
scipy.sparse.csr_matrix
scipy.sparse.eye
scipy.sparse.diags
scipy.sparse.dok_matrix
The calling methods from the pandas
pandas.read_csv
pandas.DataFrame.groupby
pandas.concat
pandas.Series.to_dict
The calling methods from the all methods
func
self.social_matrix.transpose
importlib.import_module
logging.getLogger
user_input_pos.append
self.Compute_Similarity_Euclidean.super.__init__
self.logger.addHandler
consumed_items_indices_list.np.array.astype
self._conv_weight
self._create_gcmc_embed
numpy.hstack
tensorflow.sqrt
numpy.vstack
d_mat_inv.dot.tocoo
heapq.nlargest
tensorflow.get_variable
u.train_matrix.indices.copy.tolist.remove
tensorflow.layers.Dense
numpy.arange.max
self.dataMatrix.sum.np.asarray.ravel
Compute_Similarity.compute_similarity
float
labels_list.extend
pandas.read_csv
tensorflow.matrix_diag_part
self.user_pos_test.keys
util.cython.random_choice.batch_randint_choice
tensorflow.layers.dropout
self.DAE.super.__init__
MyClass.build_graph
user_input_neg.append
self.layer.append
getattr
idx.zr_matrix.toarray
bytes.hashlib.md5.hexdigest
neg_items.T.np.reshape.tolist
self.TimeOrderPointwiseSampler.super.__init__
items_list.extend
self.dataset.to_csr_matrix.transpose
self.useOnlyBooleanInteractions
tensorflow.truncated_normal_initializer
self._get_data_path
self._attention_mlp
tensorflow.range
self._get_user_embeddings
social_neighbors_indices_list.np.array.astype
self.W_sparse.tocsr
dict
pandas.concat.groupby
self._create_variable
self.weights_p.append
numpy.cumsum
numpy.exp
tensorflow.cond
start.end.min
self.user_pos_train.keys
self.adjacient_matrix
self._dropout_sparse
os.path.basename
neg_item_list.append
self.BatchSampler.super.__init__
arg.startswith
metric_dict.items
self.lr_D.tf.train.AdamOptimizer.minimize
_generate_positive_items
arg.split
self.train_matrix.copy.copy
self.MultiVAE.super.__init__
backend.UniEvaluator
self.get_train_data
tensorflow.nn.softplus
users_list.extend
dataset.get_user_test_dict
self.ItemKNN.super.__init__
tensorflow.reduce_sum
u.train_dict.copy.index
tensorflow.Session
tensorflow.train.AdagradOptimizer.apply_gradients
dataset.trainMatrix.tocsr
self.data_source.len.np.random.permutation.tolist
item_input_pos.append
diag_vals.tf.linalg.LinearOperatorLowerTriangular.to_dense
self.lr_D.tf.train.GradientDescentOptimizer.minimize
Logger.warning
self.FISM.super.__init__
Logger
self.MultiDAE.super.__init__
self.evaluate_model
pandas.read_csv.groupby
num_idx_pos.append
f.readlines
u.train_dict.copy.remove
self.lr.tf.train.AdamOptimizer.minimize
md5_fin.readlines
self.degree_matrix
tensorflow.random_uniform_initializer
numpy.max
self.TimeOrderPairwiseSampler.super.__init__
items_list.append
coo.col.coo.row.np.asarray.transpose
tensorflow.name_scope
pandas.Series
util.learner.optimizer
tensorflow.linalg.LinearOperatorLowerTriangular
self.weights.append
self.evaluator.evaluate
iter
self.userids.items
print
line.strip
unique_user.unique_user.len.range.pd.Series.to_dict
itertools.count
users_list.append
normalized_adj_single
self.trainMatrix.keys
item_recent.append
x.ljust
evaluator.ProxyEvaluator
numpy.zeros
self.WRMF.super.__init__
self.training_generator
numpy.random.permutation
queries.get_shape
numpy.where
utils.split_by_loo
self.biases_p.append
self.CFGAN.super.__init__
self.num_items.self.num_users.np.array.astype
self.Sampler.super.__init__
self.Caser.super.__init__
item_input_recent.append
SyntaxError
self.train_tar.append
tensorflow.reduce_max
neg_items_list.extend
_sampling_negative_items.append
tensorflow.div
util.tool.csr_to_user_dict_bytime
_create_logger
learning_rate.tf.train.RMSPropOptimizer.minimize
item_seq_list.append
model.predict
next
self.DMF.super.__init__
self.pairwise_neg_sampling
tensorflow.identity
self.train_matrix.dot
item_pos_list.extend
os.getcwd
tensorflow.reshape
K_.get_shape.as_list
FileNotFoundError
ratings.append
user_items.extend
self.dataset.train_matrix.todok
tensorflow.log
tensorflow.contrib.layers.variance_scaling_initializer
self._create_placeholders
user_input.extend
consumed_items_values_list.append
tensorflow.add_n
configparser.ConfigParser
self.alg_arg.items
pandas.DataFrame
numpy.int32.sorted_items.np.array.tolist
tensorflow.Variable
self.final_act
multihead_attention
tensorflow.nn.leaky_relu
u.self.user_matrix.getrow.toarray
tensorflow.nn.softmax
tensorflow.train.AdagradOptimizer
os.remove
m.metric_dict
self._read_cmd_arg
os.path.join
tensorflow.assign
numpy.transpose
item_input_social.extend
numpy.array.sort
label_list.append
tensorflow.losses.log_loss
self.FPMCplus.super.__init__
Logger.critical
self.applyAdjustedCosine
item_list.extend
pandas.read_csv.append
tensorflow.train.AdamOptimizer
self._conv_layer
unobsv_mat.append
dataset.train_matrix.toarray
util.tool.pad_sequences
util.Configurator
self.delta_P.assign
tensorflow.scatter_update
util.cython.tools.is_ndarray
tensorflow.initializers.zeros
executor.map
tensorflow.train.MomentumOptimizer
conv_h
self.lr_G.tf.train.AdamOptimizer.minimize
tensorflow.maximum
tensorflow.nn.selu
tensorflow.contrib.layers.apply_regularization
util.log_loss
numpy.savetxt
self.IRGAN.super.__init__
self.SeqAbstractRecommender.super.__init__
init
math.ceil
batch_result.append
Logger.error
sp_mat.tocoo.astype
set
csr_to_user_dict.items
feedforward
dataset.get_user_train_dict
value.indices.copy
util.data_iterator.DataIterator
A.todok.todok
tensorflow.nn.log_softmax
self.GRU4Rec.super.__init__
tensorflow.squared_difference
random.seed
Exception
self.dataMatrix.T.dot
self.DeepICF.super.__init__
self.dataMatrix.power.sum
self.evaluate
self.max_pooling
tensorflow.train.GradientDescentOptimizer.minimize
pandas.read_csv.sort_values
DataIterator
tensorflow.nn.embedding_lookup
numpy.full
self.reduce_dimension_layer
X.tocoo.astype
scipy.sparse.diags
tensorflow.random_uniform
collections.OrderedDict
self.train_seq.append
tensorflow.nn.conv2d
numpy.repeat
distutils.core.Extension
logging.Formatter
numpy.isinf
tensorflow.variable_scope
functools.wraps
util.tool.activation_function
self.user_matrix.getrow
data.dataset.Dataset
user_id.toarray
tensorflow.SparseTensor
normalize
self.JCA.super.__init__
i.item_csr.getnnz
util.tool.get_initializer
MyClass
self.NGCF.super.__init__
list
numpy.maximum
self.user_pos_train.items
item_pos_list.append
MyClass.train_model
u_data.tolist
data.TimeOrderPairwiseSampler
self._init_weights
int
util.batch_randint_choice
numpy.power
numpy.random.seed
tensorflow.train.RMSPropOptimizer
tensorflow.contrib.layers.python.layers.batch_norm
os.makedirs
data.PointwiseSampler
numpy.array
self.dis
logging.FileHandler.setFormatter
tensorflow.floor
self.HRM.super.__init__
self._encoding
items.np.unique.tolist
tensorflow.gradients
self._load_test_neg_items
tensorflow.cast
time.time
u.train_matrix.indices.copy.tolist
self._create_ngcf_embed
item_id.self.Cui.reshape
weight_initializer
os.chdir
self.compute_similarity_object.compute_similarity
tensorflow.sequence_mask
item_input_neg.extend
self.convertDistribution
self.loss_fun
numpy.log
tensorflow.zeros_initializer
tensorflow.nn.l2_normalize
self._softmax_neg
numpy.random.rand
handler.flush
self.item_matrix.getcol
self._check_saved_data
self.dataMatrix.tocsr
self.dataset.to_csr_matrix
concurrent.futures.ThreadPoolExecutor
self._split_data
self.num_users.self.num_users.np.array.astype
NotImplementedError
item_seq_list.extend
tensorflow.transpose
item_id.self.Pui.reshape
self._flush
zero_init
tensorflow.tile
self.MF.super.__init__
numpy.arange
item_data.toarray.squeeze
self._create_pre_optimizer
max
tensorflow.shape
numpy.dot
self.PairwiseSampler.super.__init__
num_idx.append
self.dataset.get_train_interactions
tensorflow.ConfigProto
consumed_items_indices_list.append
self.normalized_adj_single
self._create_layer
tensorflow.not_equal
data.PairwiseSampler
ValueError
numpy.zeros_like
self.APR.super.__init__
format
self.logger.warning
self.dataset.time_matrix.todok
Compute_Similarity_Euclidean
self.train_matrix.copy
util.Logger
self._init_data
SequentialSampler
scipy.sparse.diags.dot
self.normalized_adj_single.tocsr
tensorflow.stop_gradient
numpy.divide
tensorflow.sign
Compute_Similarity_Python
self.__str__
tensorflow.reduce_mean
tensorflow.nn.rnn_cell.DropoutWrapper
self.batch_norm_layer
self.dataMatrix.power.sum.np.array.ravel.copy
importlib.util.find_spec
self._get_pairwise_all_data
k.str.metric.re_metric_dict.ljust
tensorflow.equal
numpy.zeros.transpose
self.norm_adj.count_nonzero
learner.lower
labels.append
util.data_generator._get_pairwise_all_likefossil_data
self._attention_MLP
precision
tensorflow.exp
self.dataMatrix_weighted.T.dot
inspect.signature.bind_partial
tensorflow.zeros_like
d_mat_inv.dot.dot
pandas.read_csv.map
rows.extend
self._load_data
self.train_matrix.todok
item_neg_list.extend
unique_item.unique_item.len.range.pd.Series.to_dict
self.fc1_ly
self.dis_layers.append
tensorflow.nn.rnn_cell.MultiRNNCell
executor.map.append
dataset.userids.keys
k.str.metric.ljust
id_end.id_start.item_input.tolist
self.avg_pooling
bound_values.arguments.items
self.SpectralCF.super.__init__
str
social_neighbors_indices_list.append
tensorflow.nn.relu
pretrained_params.append
fin.read
numpy.random.randint
tensorflow.placeholder_with_default
math.sqrt
numpy.sqrt
tensorflow.constant
out_hs.append
self.dataMatrix.power
self.SRGNN.super.__init__
recent_items_list.extend
Cython.Build.cythonize
user_id.self.Cui.T.reshape
config.params_str
tensorflow.add
self.ggnn
self._create_gcn_embed
tensorflow.nn.moments
numpy.multiply
util.argmax_top_k
isinstance
line.strip.split.extend
csr_to_user_dict
self._sample_negative
tensorflow.concat.get_shape
self.train_matrix.todense
self.logger.debug
u_data.sample.sample
_DataLoaderIter
sp_mat.tocoo
tensorflow.layers.conv1d
noise.split
embed_initializer
numpy.sum.flatten
tensorflow.contrib.layers.xavier_initializer
sys.stderr.flush
self._create_optimizer
scipy.sparse.eye
tensorflow.nn.dropout
tensorflow.matmul
tensorflow.nn.elu
util.learner.pointwise_loss
tensorflow.layers.dense
any
self._create_lightgcn_embed
self._create_adversarial
tensorflow.log_sigmoid
user_list.append
tensorflow.group
self.dataset.train_matrix.tocsc
layer.apply
logging.FileHandler
self._build_session_graph
g_h.g_l.ljust
u.train_dict.copy
sess.run
self.input_supply
self._split_A_hat_node_dropout
util.data_generator._get_pointwise_all_likefossil_data
tensorflow.truncated_normal
user_pos_dict.items
self.NAIS.super.__init__
self.logger.critical
self._create_loss
self.MLP.super.__init__
pickle.load
self.train_matrix.transpose
util.learner.pairwise_loss
self.Compute_Similarity_Python.super.__init__
tensorflow.random_normal_initializer
user_test_dict.items
numpy.argsort
numpy.ones_like
user_pos_len.append
numpy.ones
momentum.learning_rate.tf.train.MomentumOptimizer.minimize
self.biases_q.append
tensorflow.sparse_retain
numpy.mean
bias_initializer
self.grouped_user.items
len
Compute_Similarity
open
zip
inspect.signature.bind
fin.readlines
self.laplacian_matrix
self.SocialAbstractRecommender.super.__init__
tensorflow.random.truncated_normal
numpy.array.append
tensorflow.ones
pandas.concat.append
distutils.core.setup
utils.check_md5
tensorflow.gather
batch.append
tensorflow.nn.sigmoid_cross_entropy_with_logits
util.logger.Logger
self.dataMatrix.tocsc
self.create_adj_mat
tensorflow.nn.l2_loss
Logger.debug
hashlib.md5
numpy.in1d
learning_rate.tf.train.AdamOptimizer.minimize
tensorflow.abs
enumerate.append
self.UniEvaluator.super.__init__
self.generateUserEmebddingFromConsumedItems
self.dataMatrix.power.sum.np.array.ravel
numpy.matmul.append
item_input_pos.extend
self.SASRec.super.__init__
self.Fossil.super.__init__
grouped_evaluator.GroupedEvaluator
n_input.append
tensorflow.contrib.layers.l2_regularizer
tensorflow.nn.sparse_softmax_cross_entropy_with_logits
numpy.identity
self._generate_sequences
min
pandas.read_csv.unique
seq_items.np.reshape.astype
self.get_adj_mat
X.tocoo
self.delta_Q.assign
_sampling_negative_items.extend
util.csr_to_user_dict
self.LightGCN.super.__init__
scipy.sparse.dok_matrix
inputs.get_shape
os.path.exists
self.TransRec.super.__init__
tensorflow.placeholder
self.generateUserEmbeddingFromSocialNeighbors
logging.StreamHandler.setFormatter
self.NPE.super.__init__
tensorflow.multiply
numpy.random.shuffle
util.pad_sequences
self.FPMC.super.__init__
tensorflow.ones_like
util.tool.csr_to_user_dict
self.logger.error
md5_out.writelines
util.data_generator._get_pointwise_all_likefism_data
numpy.squeeze
social_neighbors_values_list.append
self.dataset.get_user_train_dict
numpy.unique
logging.StreamHandler.setLevel
os.path.isfile
data.TimeOrderPointwiseSampler
numpy.concatenate
self.hidden_act
self._split_A_hat
end.start.X.count_nonzero
pandas.Series.to_dict
user_list.extend
numpy.asarray
self._create_inference_adv
tensorflow.linalg.solve
eval
numpy.searchsorted
tensorflow.train.exponential_decay
numpy.diag
tensorflow.gather_nd
self.ProxyEvaluator.super.__init__
tensorflow.layers.Conv2D
users.tolist
self.training_discriminator
dataMatrix.copy
self.CDAE.super.__init__
num_idx_neg.append
consumed_items_values_list.np.array.astype
self._get_SocialItemsSet
self.train_matrix.todok.items
self._read_config_file
logging.StreamHandler
util.tool.randint_choice
tensorflow.get_collection
self._create_inference
user_input_id.append
learning_rate.tf.train.GradientDescentOptimizer.minimize
utils.filter_data
tensorflow.nn.dynamic_rnn
self.lr_G.tf.train.GradientDescentOptimizer.minimize
util.randint_choice
enumerate.extend
self.evaluator.metrics_info
self.graph.transpose
bat_sp_mat.tocoo.astype
self.biases.append
tensorflow.sparse_placeholder
self.p_graph
id_end.id_start.user_input.tolist
A.todok.tolil
util.DataIterator
tensorflow.stack
tensorflow.nn.tanh
self.GRU4RecPlus.super.__init__
self._get_input_all_data
tensorflow.squeeze
numpy.mat
numpy.log2
self._convert_sp_mat_to_sp_tensor.sum
super
values.extend
pandas.concat
cols.extend
GEN
RandomSampler
numpy.concatenate.append
numpy.random.choice
numpy.reshape
self.gen_layers.append
coo.col.coo.row.np.mat.transpose
u.train_matrix.indices.copy
self.logger.setLevel
sys.stdout.flush
configparser.ConfigParser.read
numpy.diff
l2_distance
u.train_dict.copy.insert
tensorflow.where
self.eval_rating_matrix
utils.split_by_ratio
tensorflow.global_variables_initializer
stacked_cell
tensorflow.square
BatchSampler
item_input.append
tensorflow.nn.rnn_cell.GRUCell
dataset.get_user_test_neg_dict
numpy.get_include
initializer
tensorflow.unique
config_sec.config.items
A_in.append
util.tool.dropout_sparse
_generative_time_order_positive_items
pos_items_list.extend
self.itemids.items
self.dropout_ly
item_input_recents.append
learning_rate.tf.train.AdagradOptimizer.minimize
tensorflow.sparse_tensor_dense_matmul
os.listdir
tensorflow.to_int32
self.create_bpr_loss
self.gen
tensorflow.norm
idx.train_matrix.toarray
adj.sum
idx.pm_matrix.toarray
line.strip.split
numpy.matmul
alias_inputs.append
self.eval_score_matrix
numpy.tile
tensorflow.pow
util.typeassert
self.sess.run
self._convert_sp_mat_to_sp_tensor
util.inner_product
logging.FileHandler.setLevel
self.DiffNet.super.__init__
util.logger.Logger.info
pandas.DataFrame.groupby
value.indices.copy.tolist
sorted
numpy.multiply.argpartition
self.SBPR.super.__init__
_sampling_negative_items
tensorflow.trainable_variables
self.ConvNCF.super.__init__
self.train_dict.items
self._session_embedding
tensorflow.random_normal
tensorflow.split
tensorflow.zeros
layer
join
tensorflow.nn.sigmoid
self._create_variables
scipy.sparse.csr_matrix
self.q_graph
self.dataset.time_matrix.todok.items
numpy.sort
numpy.sum
tensorflow.set_random_seed
pandas.read_csv.value_counts
TypeError
inspect.signature
i.self.item_matrix.getcol.toarray
n_file.split
os.path.dirname
tensorflow.expand_dims
_Dataset
self._regular
A_hat.astype.astype
self._split_A_hat.append
social_neighbors_values_list.np.array.astype
loss_function.lower
pre_trained_params.append
enumerate
self.weights_q.append
self.Pop.super.__init__
tensorflow.to_float
self.bias_variable
self._sample_neg_items
self.conv_v
numpy.atleast_2d.toarray
tensorflow.convert_to_tensor
tensorflow.losses.sigmoid_cross_entropy
tensorflow.divide
temp_embed.append
self.train_matrix.transpose.tocsr
numpy.atleast_2d
self.train_matrix.toarray
user_id.self.Pui.T.reshape
numpy.float32.result.np.array.flatten
line.strip.split.strip
Logger.info
DIS
pandas.read_csv.dropna
self.filters.append
tensorflow.train.AdagradOptimizer.minimize
pos_items.num.self.num_items.randint_choice.tolist
suk_input.extend
pandas.concat.unique
self.W_sparse.self.train_matrix.dot.toarray
util.l2_loss
param.lower
rowsum.np.power.flatten
self.RandomSampler.super.__init__
user_input.append
self._shuffle_index
self.logger.info
self.GroupedEvaluator.super.__init__
item_input_neg.append
self.P.append
numpy.eye
self.SequentialSampler.super.__init__
self.weight_variable
range
p_input.append
tensorflow.eye
self.applyPearsonCorrelation
numpy.random.binomial
self.dataMatrix.sum
bat_sp_mat.tocoo
tensorflow.sigmoid
util.data_generator._get_pairwise_all_likefism_data
numpy.linalg.eig
configparser.ConfigParser.sections
tensorflow.concat
tensorflow.layers.Dropout
self.lib_arg.items
A_out.append
self.NeuMF.super.__init__
tensorflow.train.GradientDescentOptimizer
KeyError

@developer
Could please help me check this issue?
May I pull a request to fix it?
Thank you very much.

Issue with multiple item features

I read the HRM paper under sequential recommender and got to your implementation.

I don't understand how to use multiple features for a given transition in the model.

Ex.
user1 movei1, genre1, timestamp1
user1 movei2 genre2, timestamp2
user1, movie3, genre2, timestamp3
user1, movie4, genre2, timestamp4
.
.
user2, movie1, genre1, timestamp1
user2, movei2 genre2, timestamp2
.
.

In file HoldOutDataSplitter.py line 28, it gets the "useridx, itemidx,rating, time".

Hope to hear back.

BPRMF unavailable?

I'm not able to find the BPRMF recommender in the "General Recommender" section? Is it unavailable for now? If so, can I contribute in developing it?

error:当采用pointwise损失,并将负采样设置为4时,recall@20出现1.0的情况

您好, 我在测试这些方法时,在ml-100k数据集上,使用loo划分,99个测试负采样。训练时的负采样设置为4,采用cross_entropy损失。得到了以下结果:
image
recall@20达到了1.0,而recall@10的值却很低。由于之前遇到过这种情况,当时是由于topK排序算法在排序过程中,对于相同的值会默认选择第一个,当模型得到的预测值非常接近且目标物品在第一个时就会出现hr,recall=1.0的情况。但是我看了下这部分的代码,由于编码能力有限,我先将evaluator.uni_evaluator.py中的evaluate function中的
candidate_items = [list(self.user_pos_test[u]) + self.user_neg_test[u] for u in batch_users]
改成了
candidate_items = [self.user_neg_test[u] +list(self.user_pos_test[u]) for u in batch_users]
这在我之前碰到的情况中是有效的。但这里他没有影响。
image

@@@而这个地方的test_items我认为两种情况下应该是一样的?这个是指g_item(目标物品)么?为什么当指定负测试样本时,test_items是这样的呢?我搞不太清楚。tool.argmax_top_k我也搞不太懂。非常希望得到您的解答,或者一个提示。非常感谢

怎么样提高模型评估阶段的速度

在这个代码中,模型每次迭代后,都会计算模型的指标。但是好像速度很慢,我的用户和产品都是10K+,用MF模型训练需要100s,但是每次计算指标都需要700s

LightGCN @20 =0.98165911 at epoch 1

LightGCN @20 =0.98165911 at epoch 1

Properties are set to the following:

[default]
######## model
recommender=LightGCN
config_dir=./conf
gpu_id=0
gpu_mem=0.5
data.input.path=dataset
data.input.dataset=gowalla
data.column.format=UI
data.convert.separator=','
user_min=0
item_min=0
splitter=given
ratio=0.8
by_time=False
metric=["Precision", "Recall"]
topk=[20]
group_view=None
rec.evaluate.neg=1
test_batch_size= 2048
num_thread=8

[hyperparameters]
lr = 0.001
reg = 1e-4
embed_size = 64
n_layers = 3
batch_size = 2048
epochs = 800
n_fold = 100
adj_type = pre

Something about Carse -- code & training speed

(1) in util/Tool.py #69 :
def csr_to_user_dict_bytime(time_matrix,train_matrix):
but, in Caser.py:
model/sequential_recommender/Caser.py line #127:
self.user_pos_train = csr_to_user_dict_bytime(self.dataset.train_matrix, self.dataset.time_matrix)

the order of inputs is not correct.

(2) The GivenData.py can't be directly used for Caser.py.

(3) The training is every slow , maybe since the negative sampling methods. Is there any method to speed up training?

(4) It would be better, giving some examples for using sequential models.

(5) Another thing:
Dose the sequential_recommender models have been tested and could them get similar performance in orignal paper?

[Suggestion] make NeuRec portable as a package instead of static project style

Issue

NeuRec is an awesome project, and I had applied TransRec session-based recommender and achieved the best performance comparing to other individual algorithms.

However, regarding the engineering perspective, NeuRec is unfriendly in terms of Python project style when trying to modify its code (e.g. add serialization feature) in order to use it as a package along with other machine learning libraries, because it does not put all the classes under a unified package namespace (normal Python package will create its own root namespace, e.g. import tensorflow as tf) and using some absolute importing for relative directory (see below log).

➜  tmp ls NeuRec
NeuRec.properties          appveyor.yml               conf                       evaluator                  next.png                   travis_linux_steps.sh
README.md                  appveyor_windows_steps.ps1 data                       main.py                    requirements.txt           tutorial.ipynb
WeChat_QR_Code.jpg         architecture.jpg           dataset                    model                      setup.py                   util
➜  tmp python
Python 3.6.3 (default, Jan 29 2020, 11:15:24)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import NeuRec.data
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dev/tmp/NeuRec/data/__init__.py", line 1, in <module>
    from .sampler import PointwiseSampler
  File "/Users/dev/tmp/NeuRec/data/sampler.py", line 4, in <module>
    from util import DataIterator
ModuleNotFoundError: No module named 'util'.
>>>

This should be done by relative importing from .util import Data or absolute importing from NeuRec.util import Data.

Solution

The solution should be easy, create a new root directory 'NeuRec' and move all components into 'NeuRec' as the subdirectories, although lots of trivial importing statements in all Python files needs to be fixed.
I have a temporary solution https://github.com/DavidChenLondon/NeuRec/tree/smk which works in our recommendation engine, and includes other features like serializing NeuRec TensorFlow models.

However, I know that NeuRec is under active development (https://github.com/wubinzzu/NeuRec/tree/v3.x with https://github.com/ZhongchuanSun/reckit) currently, so it's hard for me to make a diverged change without discussing with NeuRec team first.

在MovieLen-1m数据集上跑GRU4Rec,统计指标异常

数据集:MovieLen-1m原始数据

Recommender:GRU4Rec

Hyperparameters:
lr=0.001
reg=0.0
layers=[100]
batch_size=256
loss=bpr
hidden_act=tanh
final_act=linear
epochs=100

rec.evaluate.neg=100

统计指标:
HR@1 HR@5 HR@10 NDCG@1 NDCG@5 NDCG@10 MRR@1 MRR@5 MRR@10
epoch 0: 0.36889184 0.71459335 0.83236706 0.36889184 0.55202842 0.59043419 0.36889184 0.49796271 0.51400179

明显高于已有文献的统计数值,望解答

uni_evaluator.py文件中的疑问

在evaluator/backend/cpp/uni_evaluator.py 文件的第138-142行

# set the ranking scores of training items to -inf,
# then the training items will be sorted at the end of the ranking list.
for idx, user in enumerate(batch_users):
    train_items = self.user_pos_train[user]
    ranking_score[idx][train_items] = -np.inf

为什么需要将训练集中的产品加入到模型的评估阶段?
当测试冷启动问题时,用户训练集中无对应的产品,是否应该考虑该情况?

SASRec

The SASRec model has no build_graph function.

Expected behaviour
For the build_graph function to be called for the SASRec class.

Actual behaviour
The build_graph function is not called because it does not exist in the SASRec class.

Suggested action
Implement the build_graph function.

Screenshot_2019-11-06 Untitled

evaluation methods and NeuMF in NeuRec3.X

I find the evaluation always run for every user. do you have some example for evaluation on sampled item?

I know that this metrics have some bias, but in some paper still use this.
image

and why there is no NeuMF implements in NeuRec3.X? is too slow to evaluation on all negative items?

NGCF index out of bounds

NGCF produces an index out of bounds error on line:

weight_initializer([self.weight_size_list[k], self.weight_size_list[k+1]]), name='W_gc_%d' % k)

IndexError: index 2 is out of bounds for axis 0 with size 2

Screenshot_2019-11-06 Untitled(2)

Properties are set to the following:

[DEFAULT]
data.input.path=neurec
data.input.dataset=ml-100k.rating
data.column.format=UIRT
data.convert.separator='\t'
data.convert.binarize.threshold=0
data.splitter=loo
data.splitterratio=[0.8,0.2]|float
rec.number.thread=20
rec.evaluate.neg=0
recommender=ngcf
topk=[10, 20, 30, 40]|int
gpu_id=0

epochs=500
batch_size=512
embedding_size=16
layer_size=[16,16]|int
learning_rate=0.001
node_dropout_flag=False
adj_type=norm
alg_type=ngcf
loss_function=BPR
learner=adam
reg=0.0
node_dropout_ratio=0.1
mess_dropout_ratio=0.1
#tnormal: truncated_normal_initializer, uniform: random_uniform_initializer,
#normal: random_normal_initializer, xavier_normal, xavier_uniform, 
#he_normal, he_uniform. Defualt: tnormal
embed_init_method=xavier_normal
weight_init_method=xavier_normal
stddev=0.01
verbose=1

DMF userid referenced before assignment

Error when running DMF model:
UnboundLocalError: local variable 'userid' referenced before assignment

The userid variable has not be assigned before its use on line 133 of DMF

Expected behaviour
The predict function to run successfully without errors.

Actual behaviour
The predict function does not run successfully with an error being generated.

Screenshot from 2019-11-06 11-44-13

Note: the image shows line 142, this is mapped to line 133 in this repo.

how to fix the error?

hi,dear
When I
$ python3 setup.py build_ext --inplace

Warning: passing language='c++' to cythonize() is deprecated. Instead, put "# distutils: language=c++" in your .pyx or .pxd file(s)
Compiling apt_evaluate_foldout.pyx because it depends on /data/logs/xulm1/myconda/lib/python3.7/site-packages/Cython/Includes/cpython/buffer.pxd.
[1/1] Cythonizing apt_evaluate_foldout.pyx
/data/logs/xulm1/myconda/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /data/logs/xulm1/LightGCN/evaluator/cpp/apt_evaluate_foldout.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'apt_evaluate_foldout' extension
gcc -pthread -B /data/logs/xulm1/myconda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/logs/xulm1/myconda/lib/python3.7/site-packages/numpy/core/include -I/data/logs/xulm1/myconda/include/python3.7m -c apt_evaluate_foldout.cpp -o build/temp.linux-x86_64-3.7/apt_evaluate_foldout.o -std=c++11
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
error: command 'gcc' failed with exit status 1

help me,please

您好,请问代码还会继续修改吗?发现些问题

您好,我目前试了下您提供的方法中的四个,发现有两个问题:
1、最基本的MF是可以运行的,但是运行CFGAN、NeuMF时都会有log文件不存在的问题,应该是在运行有的方法时无法自动生成log文件。
2、我还运行了IRGAN的代码。发现程序的矩阵维数设置应该有问题,错误提示如下:
File "D:\MyProject\NeuRec-master\NeuRec-master\model\item_ranking\IRGAN.py", line 193, in get_train_data_one_user neg = np.random.choice(self.all_items, size=len(pos), p=prob) File "mtrand.pyx", line 1142, in mtrand.RandomState.choice ValueError: a and p must have same size
即在IRGAN.py文件的193行的 neg = np.random.choice(self.all_items, size=len(pos), p=prob)语句发生错误。
不知道您能否抽出时间查看一下,如果是我自己配置的问题希望您能指点一下,万分感谢!

error:当采用pointwise损失,并将负采样设置为4时,recall@20出现1.0的情况

您好, 我在测试这些方法时,在ml-100k数据集上,使用loo划分,99个测试负采样。训练时的负采样设置为4,采用cross_entropy损失。得到了以下结果:
image
recall@20达到了1.0,而recall@10的值却很低。由于之前遇到过这种情况,当时是由于topK排序算法在排序过程中,对于相同的值会默认选择第一个,当模型得到的预测值非常接近且目标物品在第一个时就会出现hr,recall=1.0的情况。但是我看了下这部分的代码,由于编码能力有限,我先将evaluator.uni_evaluator.py中的evaluate function中的
candidate_items = [list(self.user_pos_test[u]) + self.user_neg_test[u] for u in batch_users]
改成了
candidate_items = [self.user_neg_test[u] +list(self.user_pos_test[u]) for u in batch_users]
这在我之前碰到的情况中是有效的。但这里他没有影响。
image

而这个地方的test_items我认为两种情况下应该是一样的?这个是指g_item(目标物品)么?为什么当指定负测试样本时,test_items是这样的呢?我搞不太清楚。tool.argmax_top_k我也搞不太懂。非常希望得到您的解答,或者一个提示。非常感谢

MacOS error: command '/usr/bin/clang' failed with exit code 1

Hi. I'm using

Python 3.9.6
tensorflow 2.5.0
numpy 1.19.5
pandas 1.3.0

When I use this command
python3 setup.py build_ext --inplace

I got this.

Warning: passing language='c++' to cythonize() is deprecated. Instead, put "# distutils: language=c++" in your .pyx or .pxd file(s)
running build_ext
building 'cpp_evaluator' extension
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I /usr/local/lib/python2.7/dist-packages/numpy/core/include -I /usr/local/lib/python3.7/dist-packages/numpy/core/include/numpy/ -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c cpp_evaluator.cpp -o build/temp.macosx-11-x86_64-3.9/cpp_evaluator.o -std=c++11
cpp_evaluator.cpp:653:10: fatal error: 'numpy/arrayobject.h' file not found
#include "numpy/arrayobject.h"
         ^~~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1

How can I fix this?

How to actually get a recommendation

Hi I was wondering, how do I actually get a list of item recommendations once the model is trained? I only see the method predict that gives a list of rating but the values are not in the range 1 to 5 (I used the ml-100k dataset).
Does any example where the model is used after being trained exist?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.