deepshift-labs / magist-algorithm Goto Github PK

Description
If I try downloading, for example, images of cars first and then images of trucks in the same Python instance with the same instance of the class, it says that it finished downloading but never actually downloads anything.

To Reproduce
Steps to reproduce the behavior:

1. Run this:

from MAGIST.Utils.WebScraper.google import GoogleScraper

scraper = GoogleScraper("config.json")

scraper.download_raw_img_dataset("art", 10, "Data/")

scraper.download_raw_img_dataset("language", 10, "Data/")

scraper.download_raw_img_dataset("cabinet hardware", 10, "Data/")

This downloads 13 images of art and 0 images of the other categories.

Expected behavior
It should ideally finish downloading the first one and start downloading the second category.

Additional context
For some weird reason, it also downloads a few more images than I require. For example, it might download 13 images instead of 10.

Issue:
Google Reverse Image Search(RIS) is incredibly inaccurate. It assumes that any image with black is night, language, or art. It rarely gets classes accurate even if the clustering was done perfectly.

Solution
Bing(via Azure) has an API to perform a Visual Search of an image. I can use a little scraping and JSON parsing to find the label. This, however, requires a billing setup that excluded it from the available options until now.

Other Alternatives
I can also cross-reference the NLP data with the Vision data to remove extraneous data. This is extremely complex making it hard to implement.

Description
The Google Image Search API was updated to have a limit of 100 images per query. This means that consecutively downloading, for example, 200 images will fail.

To Reproduce
To reproduce the behavior, run the following code:

from MAGIST.Utils.WebScraper.google import GoogleScraper

g = GoogleScraper("config.json")
g.download_raw_img_dataset("cars", 200, "Data/")

Expected behavior
It should ideally download 200 images and verify their integrity before downloading.

Screenshots

Additional context
I have teamed up with @arrrlo who made the GoogleImageSearch module for Python with the issue also posted on @arrrlo's repo.

This requires the module pyparsing==2.4.2 which is part of a circular dependency issue(#4 ).

Problem and Description
For the General Intelligence, the microphone transcription should always be running. But, if that is always in the queue, the other tasks like unsupervised detection and data download won't occur. The primary and daemon thread will be occupied. Is it possible to add a second worker node and a manager to handle task distribution and processing?

Solution
If we add a second __worker function to the PriorityQueue class as well as a manager to automatically distribute tasks once they are published, we can have more processes running with faster and more efficient execution. This also resolves the issue described above.

Alternatives
Multi-threading and Multi-processing managers were considered. In this idea, there would be a multiprocessing pool that would take tasks and assign them to a core manager. Each core manager takes the tasks and assigns them to a thread. This way, we get complete resource utilization. However, Python's GIL(Global Interpreter Lock) makes it arduous to implement. There is a separate issue for this: #10 .

Description
Occasionally, the Microphone Transcriber freezes after printing:

GoogleAudioTranscriber Listening...

I am not really sure what causes it or what can be done.

To Reproduce
Not sure when and where it happens. My theory is that some process doesn't close correctly and occupies the microphone.

Expected behavior
Ideally, it should finish transcribing or provide a microphone-occupied error.

Screenshots
There isn't much to show but I do get these ALSA errors:

ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card

Description
The NeuralDB is currently hard-coded to have several different fields of collections that it can write to and query from. This seems like a rigid system that might be hard to change when more input fields and sensors are added. The hard-coded nature also makes it impractical for live applications

Potential solutions
I am not really sure yet. If the community has any ideas, they are welcome to post a comment below. One way would be to make a handler that dynamically makes new fields and functions upon request. This would involve using macro-coding or polymorphic code that updates its own methods as new fields emerge. I doubt Python can do this.

Perhaps if I had a template function that modified its parameters as new fields and collections were added.

Description
The MAGIST CNN isn't tuned properly for large and high-class datasets. More tuning is required for higher convergence and lower training time.

Expectation
Ideally, it should train within 30mins to be effective.

Additional context
Need to apply TensorFlow transfer learning API for custom AI models. NOTE: This is not the same thing as the TF Object Detection API.

Problem and Solution Description
Right now, MAGIST occupies only 2 threads, namely the main thread and the daemon thread. This is problematic when the AI is simultaneously training and processing data. This makes it nearly impossible to run without major improvements. If we had a manager that can assign tasks to all the threads across all cores in the system, it could run faster and more efficiently.

Detailed Description
Multi-threading and Multi-processing managers can be used to resolve this. In this, there would be a multiprocessing pool that would take tasks and assign them to a core manager. Each core manager takes the tasks and assigns them to a thread. This way, we get complete resource utilization. However, Python's GIL(Global Interpreter Lock) makes it arduous to implement.

Alternatives
A much simpler idea would be to implement a second worker node in the PriorityQueue class. If we add a second __worker function to the PriorityQueue class as well as a manager to automatically distribute tasks once they are published, we can have more processes running with faster and more efficient execution. This also resolves the issue described in #9 .

Is your feature request related to a problem? Please describe.
Running the entire algorithm on a single computer is impractical. If often crashes my entire computer. Since MAGIST is multi-agent, is it possible to do microtask load distribution?

Describe the solution you'd like
Using some type of high-speed WiFi communication protocol, send microtasks to different computers and they will respond back. Maybe create a shared network drive to store large amounts of data.

Describe alternatives you've considered
I doubt there are many other alternatives besides getting a more powerful computer.

Description
The clustering algorithm leaves the parts of the masked images that aren't part of the clustered class black. This makes it incredibly hard for Google to classify them. Using some clever algorithm to remove these black spots can help a lot.

To Reproduce
Steps to reproduce the behavior:

1. Put this image in your script directory:
   

2. Run this:

from MAGIST.Vision.UnsupervisedModels.img_cluster import RoughCluster

cluster = RoughCluster("config.json")

imgs = cluster.unsupervised_clusters(6, "Input.jpg", (150, 150), "Clusters")


from MAGIST.Utils.WebScraper.google import GoogleScraper

scraper = GoogleScraper("config.json")

labels = []

for i in imgs:
	label = scraper.reverse_image_search(i)
	labels.append(label)

print(labels)

Expected behavior
Ideally, it should mask out nearby, similar pixels but it doesn't. This leaves massive, unstructured black gaps that cause a lot of issues when reverse searching.

Screenshots

Additional context
Google generally returns language or night when there is too much black. The solution would be something like this:

1. Find each masked image's edge pixels.
2. Compute all possible lines that can be formed from edge pixels.
3. Crop image at that line if all pixels in the line are black.

This is super computationally intensive, however.

There are several dependency conflicts with pyparsing, apache-beam, requests, urllib, etc.

To Reproduce
Steps to reproduce the behavior:

1. Activate the environment: conda activate MAGIST_AI or source path/to/venv/bin/activate
2. Run the dependency checker: pip3 check

Expected behavior
Ideally, there should be no output, instead, I get a bunch of dependency conflicts.

Screenshots

$ pip3 check
object-detection 0.1 requires tf-models-official, which is not installed.
selenium 4.1.2 has requirement urllib3[secure,socks]~=1.26, but you have urllib3 1.25.11.
sacrebleu 2.0.0 has requirement tabulate>=0.8.9, but you have tabulate 0.8.7.
requests 2.24.0 has requirement idna<3,>=2.5, but you have idna 3.3.
pylint 2.12.2 has requirement mccabe<0.7,>=0.6, but you have mccabe 0.7.0.
google-images-search 1.4.2 has requirement click~=7.0, but you have click 8.1.2.
apache-beam 2.37.0 has requirement numpy<1.22.0,>=1.14.3, but you have numpy 1.22.3.

Additional context
To run TensorFlow, I had to downgrade pyparsing==2.4.2.

Description
As of now, the Self-Attention Mechanism is using a simple Cosine Similarity/Dot Product Similarity Algorithm with a selectivity threshold. This is proven to be very inaccurate and cannot be reverse searched as the incorrect selections will invalidate the integrity of the database.

To Reproduce
To reproduce the behavior:

from MAGIST.NLP.SelfAttention import TextPreprocessing

t = TextPreprocessing("config.json")

out = t.__call__("Hello, my name is John. I am a dummy script.")

for i in out:
    print(i)

Output

[5.967583262815608, 'hello', 'Good']
[3.7432159225461947, 'my', 'Not']
[2.520566459677965, 'name', 'Not']           ---> Incorrect; This should be "Good"
[5.6983463875519735, 'is', 'Not']
[4.848795399908668, 'john', 'Not']
[6.083478457022617, 'i', 'Good']
[9.443521265161667, 'am', 'Good']
[8.284217064260607, 'a', 'Good']             ---> Incorrect; This should be "Not"
[8.485852410408823, 'dummy', 'Good']
[2.466104715281189, 'script', 'Not']         ---> Incorrect; This should be "Good"

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
This was expected since this algorithm is very primitive. Perhaps, a better positional embedding or an end-to-end LSTM-Dense neural network would improve its performance.

Is your feature request related to a problem? Please describe.
This program needs all the efficiency it can get and since the latest version of Python boasts much faster execution, it is worth switching to. It is not an immediate problem but is important as the project progresses.

Describe the solution you'd like
Most of MAGIST's features support the latest Python version, but some libraries may use deprecated methods or features. Those issues should be resolved.