Comments (2)
Just for more clarity so for example i have saved set_count_1.h5 model for the first set and similarly i want to test more data with the help of this model for the same set. but I am not able to produce a script.
from aes_dl.
I have worked on a script, let me know if it make sense:
`import time
import torch
import transformers as ppb
import warnings
warnings.filterwarnings('ignore')
cuda = torch.device('cuda')
For DistilBERT:
model_class, tokenizer_class, pretrained_weights = (ppb.DistilBertModel, ppb.DistilBertTokenizer, 'distilbert-base-uncased')
tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
model = model_class.from_pretrained(pretrained_weights)
with torch.cuda.device(cuda):
test_essays = demo_df['essay']
sentences = []
tokenize_sentences = []
train_bert_embeddings = []
tokenized_test = test_essays.apply((lambda x: tokenizer.encode(x, add_special_tokens=True ,max_length=200)))
max_len = 0
for i in tokenized_test.values:
if len(i) > max_len:
max_len = len(i)
padded_test = np.array([i + [0]*(max_len-len(i)) for i in tokenized_test.values])
attention_mask_test = np.where(padded_test != 0, 1, 0)
test_input_ids = torch.tensor(padded_test)
test_attention_mask = torch.tensor(attention_mask_test)
with torch.no_grad():
last_hidden_states_test = model(test_input_ids, attention_mask=test_attention_mask)
test_features = last_hidden_states_test[0][:,0,:].numpy()
train_x,train_y = train_features.shape
test_x,test_y = test_features.shape
testDataVectors = np.reshape(test_features,(test_x,1,test_y))
lstm_model.load_weights("./model_weights/final_lstm1.h5")
preds = lstm_model.predict(testDataVectors)
print(int(np.around(preds)))
`
from aes_dl.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aes_dl.