follicles_detection's Issues
measure of computation time for random_classifier submission
Test réalisé
Je lance la prédiction pour une image (D-1M06-3.jpg) avec le détecteur actuel
(random windows + classifier). Je mesure le temps d'exécution
ligne par ligne de la prédiction dans deux situations:
- A: 200 boîtes de 1000px
- B: 5000 boîtes de 200px
Note: cette image mesure 14_207 × 11_354 px
. Si l'on veut couvrir
l'ensemble de cette image d'une grille de fenêtres de 200 x 200 px
(taille approximative des follicules primaires) il en faut ~4_000.
(les fenêtres ne se recouvrent pas donc on rate beaucoup d'objets).
Résultats
Si on réalise des prédictions sur les deux échelles de grilles
décrite ci-dessous, le temps de prédiction est ~6 min / image.
Lors de l'exécution de la commande ramp-test
il y a:
- prédiction pour 40 images
- 5 folds
- donc un total de ~20h uniquement pour la prédiction
A: peu de grandes boîtes
(mesuré sur macbook pro benjamin, pas d'utilisation GPU)
200 boîtes de 1000px
Temps total: 18.6s dont:
- 0.0s
load_image
- 2.2s
image.crop(box) x 200
- 1.7s
cropped_image.resize(224, 224) x 200
- 14.5s
model.predict(resized_image) x 200
B: beaucoup de petites boîtes
(mesuré sur macbook pro benjamin, pas d'utilisation GPU)
5000 boîtes de 200px
Temps total: 351s (5min 51s) dont:
- 0.0s
load_image
- 2.6s
image.crop(box) x 5000
- 4.4s
cropped_image.resize(224, 224) x 5000
- 341s
model.predict(resized_image) x 5000
Résultats bruts
A
Timer unit: 1 s
Total time: 16.3381 s
File: submissions/random_classifier/object_detector.py
Function: predict_image at line 18
Line # Hits Time Per Hit % Time Line Contents
==============================================================
18 def predict_image(image, model):
19 category_to_label = {
20 201 0.0 0.0 0.0 0: "Negative",
21 201 0.0 0.0 0.0 1: "Primordial",
22 201 0.0 0.0 0.0 2: "Primary",
23 201 0.0 0.0 0.0 3: "Secondary",
24 201 0.0 0.0 0.0 4: "Tertiary",
25 }
26
27 201 1.7 0.0 10.5 image = image.resize((224, 224))
28 201 0.0 0.0 0.3 image = np.array(image)
29 201 0.1 0.0 0.4 image = tf.reshape(image, (1, 224, 224, 3))
30 201 14.5 0.1 88.7 pred = model.predict(image)
31 201 0.0 0.0 0.1 predicted_category, proba = np.argmax(pred), np.max(pred)
32 201 0.0 0.0 0.0 predicted_label = category_to_label[predicted_category]
33 201 0.0 0.0 0.0 return predicted_label, proba
Total time: 18.5689 s
File: submissions/random_classifier/object_detector.py
Function: predict_locations_for_windows at line 56
Line # Hits Time Per Hit % Time Line Contents
==============================================================
56 def predict_locations_for_windows(coupe, model, window_size=1000, num_windows=200):
57 1 0.0 0.0 0.0 boxes = generate_random_windows_for_image(
58 1 0.0 0.0 0.0 coupe, window_size=window_size, num_windows=num_windows
59 )
60 1 0.0 0.0 0.0 predicted_locations = []
61 202 0.0 0.0 0.1 for box in boxes:
62 201 2.2 0.0 11.9 cropped_image = coupe.crop(box)
63 201 16.3 0.1 88.0 label, proba = predict_image(cropped_image, model)
64 201 0.0 0.0 0.0 if label != "Negative":
65 2 0.0 0.0 0.0 predicted_locations.append({"bbox": box, "label": label, "proba": proba})
66 1 0.0 0.0 0.0 return predicted_locations
Total time: 18.5832 s
File: submissions/random_classifier/object_detector.py
Function: predict at line 82
Line # Hits Time Per Hit % Time Line Contents
==============================================================
82 @do_profile(follow=[predict_locations_for_windows, predict_image])
83 def predict(self, X):
84 1 0.0 0.0 0.0 print(f"Predicting on images ({len(X)})")
85 # X = numpy array N rows, 1 column, type object
86 # each row = one file name for an image
87 1 0.0 0.0 0.0 all_predictions = []
88 2 0.0 0.0 0.0 for i, image_path in enumerate(X):
89 # TEMP: only make prediction for first image
90 1 0.0 0.0 0.0 if i == 0:
91 1 0.0 0.0 0.0 print(f" {image_path}")
92 1 0.0 0.0 0.1 img = load_image(image_path)
93 1 18.6 18.6 99.9 pred_list = predict_locations_for_windows(img, self._model)
94 1 0.0 0.0 0.0 all_predictions.append(pred_list)
95 else:
96 all_predictions.append([])
97
98 1 0.0 0.0 0.0 y_pred = np.empty(len(X), dtype=object)
99 1 0.0 0.0 0.0 y_pred[:] = all_predictions
100 1 0.0 0.0 0.0 return y_pred
B
Total time: 347.742 s
File: submissions/random_classifier/object_detector.py
Function: predict_image at line 18
Line # Hits Time Per Hit % Time Line Contents
==============================================================
18 def predict_image(image, model):
19 category_to_label = {
20 5001 0.0 0.0 0.0 0: "Negative",
21 5001 0.0 0.0 0.0 1: "Primordial",
22 5001 0.0 0.0 0.0 2: "Primary",
23 5001 0.0 0.0 0.0 3: "Secondary",
24 5001 0.0 0.0 0.0 4: "Tertiary",
25 }
26
27 5001 4.4 0.0 1.3 image = image.resize((224, 224))
28 5001 1.0 0.0 0.3 image = np.array(image)
29 5001 1.4 0.0 0.4 image = tf.reshape(image, (1, 224, 224, 3))
30 5001 340.6 0.1 97.9 pred = model.predict(image)
31 5001 0.3 0.0 0.1 predicted_category, proba = np.argmax(pred), np.max(pred)
32 5001 0.0 0.0 0.0 predicted_label = category_to_label[predicted_category]
33 5001 0.0 0.0 0.0 return predicted_label, proba
Total time: 350.644 s
File: submissions/random_classifier/object_detector.py
Function: predict_locations_for_windows at line 56
Line # Hits Time Per Hit % Time Line Contents
==============================================================
56 def predict_locations_for_windows(coupe, model, window_size=200, num_windows=5000):
57 1 0.0 0.0 0.0 boxes = generate_random_windows_for_image(
58 1 0.0 0.0 0.0 coupe, window_size=window_size, num_windows=num_windows
59 )
60 1 0.0 0.0 0.0 predicted_locations = []
61 5002 0.3 0.0 0.1 for box in boxes:
62 5001 2.6 0.0 0.7 cropped_image = coupe.crop(box)
63 5001 347.8 0.1 99.2 label, proba = predict_image(cropped_image, model)
64 5001 0.0 0.0 0.0 if label != "Negative":
65 193 0.0 0.0 0.0 predicted_locations.append({"bbox": box, "label": label, "proba": proba})
66 1 0.0 0.0 0.0 return predicted_locations
Total time: 350.674 s
File: submissions/random_classifier/object_detector.py
Function: predict at line 82
Line # Hits Time Per Hit % Time Line Contents
==============================================================
82 @do_profile(follow=[predict_locations_for_windows, predict_image])
83 def predict(self, X):
84 1 0.0 0.0 0.0 print(f"Predicting on images ({len(X)})")
85 # X = numpy array N rows, 1 column, type object
86 # each row = one file name for an image
87 1 0.0 0.0 0.0 all_predictions = []
88 2 0.0 0.0 0.0 for i, image_path in enumerate(X):
89 # TEMP: only make prediction for first image
90 1 0.0 0.0 0.0 if i == 0:
91 1 0.0 0.0 0.0 print(f" {image_path}")
92 1 0.0 0.0 0.0 img = load_image(image_path)
93 1 350.7 350.7 100.0 pred_list = predict_locations_for_windows(img, self._model)
94 1 0.0 0.0 0.0 all_predictions.append(pred_list)
95 else:
96 all_predictions.append([])
97
98 1 0.0 0.0 0.0 y_pred = np.empty(len(X), dtype=object)
99 1 0.0 0.0 0.0 y_pred[:] = all_predictions
100 1 0.0 0.0 0.0 return y_pred
formaliser la prédiction dans un exemple de soumission
fichiers à créer:
start_predictions.py
submissions/
sliding_window/
object_detector.py
# object_detector.py
class ObjectDetector:
def __init__(self):
pass
def fit(self, X, y):
# ignore inputs
# load trained model from file
self._model = load("xxxx")
return self
def predict(self, X):
# X = numpy array N rows, 1 colum, type object
# each row = one file name for an image
# pour chaque image:
# charger
# découper en grille
# faire une prédiction
pred = self._model.predict(cropped_image)
all_predictions.append(pred)
# mettre en forme all_predictions:
# un numy array N rows, 1 column
# chaque element = une liste qui correspondent aux prédictions pour CETTE image
# une prédiction = {bbox, label, proba}
return all_predictions
# start_predictions.py
from submissions.xx.object_detecor import ObjectDetector
detector = ObjectDetector()
detector.fit(None, None)
X = ["DIMCXXX.jpg", "xxxx.jp"]
predictions = detector.predict(X)
Organisation du code:
- bien séparer les fonctions de la classe des fonctions utilitaires
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.