MySQL 8.0.12: The Devil Is In The Detail

More than a billion views now use MySQL to access post, comment, member, and more. As we have evolved the product and added new functionality, the underlying technologies that power OpenSooq have changed substantially. We have dozens of background tasks that handle things like payments and analytics. Many of the queries that are run by these tasks are expensive SELECTs, and we were starting to notice an impact on frontend performance. 

To help improve OpenSooq even more, we

Continue Reading

Persisting Trained Machine Learning Models Using MsgPack

Introduction

At OpenSooq, we take AI seriously. Tensroflow and Scikit-Learn are two examples of popular packages used for machine learning, that are used in OpenSooq for different aspects of our e-commerce platform and its supporting services and moderation tools. In Supervised Machine Learning there are two phases training (fitting the model) and prediction. Training is done offline, then models get evaluated and then persisted to be deployed on production. Tensroflow models can be persisted as frozen protocol buffer files, on

Continue Reading

Building Data Visualization for SolrCloud 7.0 using Apache Zeppelin

Data Visualization allow you to natively interact with all your data via custom dashboards. Apache Zeppelin is a new and incubating multi-purposed web-based notebook which brings data ingestion, data exploration, visualization, sharing and collaboration features to Hadoop, Solr and Spark.

Apache Zeppelin turns your data into informative dashboards and reports that are easy to read, easy to share, and fully customizable.

Multi-purpose Notebook 

The Notebook is the place for all your needs

  1.  Data Ingestion
  2.  Data Discovery
  3.  Data Analytics
  4.  Data
Continue Reading

Opensooq Gloom : URL Shortener Service

Opensooq Gloom is an opensource service that takes long URLs and squeezes them into fewer characters to make a link that is easier to share, tweet, or email to friends. Users can create these short links through the web interface, or they can programatically create them through the API. With the Gloom you can write applications that use simple HTTP methods

Continue Reading

On-the-fly ad-hoc docker compose development stack

The problem

In this blog post we are going to discuss an application stack that uses MongoDB, Redis and NodeJS. We want a single docker compose command that launches the entire stack, still we want to the developer be able to use his/her favorite IDE to edit the source code without having to rebuild docker images. We call this on-the-fly ad-hoc stack because it does not involve building any docker images or hosting a docker registry, the reason for this

Continue Reading

NGINX TCP/UDP Load Balancer with Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. It is the soul of web, it’s capable of turning almost any application into a speed-demon.

But what happens sometime that you can not afford the 20-30GB RAM Servers to cache your data. Also, it is not a smart idea to depend on a single node to cache all your data (single point of failure).

In this article,

Continue Reading

Using SO_REUSEPORT Flag In Multi-Processes UDP Server

SO_REUSEPORT is a new flag available in Linux 3.9+ that allows several threads to bind the same port simultaneously instead of having to race for the same socket, having the kernel round-robin between them. And this was intended to improve the performance of multi-threaded network server applications running on top of multi-core systems.

The real application of this feature should be when sometimes a worker is stalled by a blocking operation, which does not only affect connections that the worker

Continue Reading

Manage your Docker image layers with Ansible

If you are managing infrastructure at large scale as we do in OpenSooq you had better have good level of automation and orchestration, your SysAdmins and operators had better handle their infrastructure as code. In OpenSooq, our preferred tool is Ansible.

Ansible is exceptionally great to put your servers into the desired state, which make it ideal for deployment orchestration and configuration management. For containerized environment, typically people use Dockerfiles to describe their setup inside the container, so they

Continue Reading

Replicating your SQL into NoSQL and full text search engine (Solr/ElasticSearch)

Opensooq uses so many open source technologies, because each piece has it’s own use case, for example we use Apache Solr to power the content moderation dashboard.

When your main data source is a strict classical ACID SQL, and you want to take that data to some other backend like NoSQL’s, ElasticSearch, or Solr as in our case, for sure you will get out of sync data. Keeping them in-sync becomes harder and

Continue Reading

Integrating Apache Solr with Play Framework in Scala

Typical IT landscape of any large size organization comprises of many different technologies. Apache Solr provides excellent ways to gather information from different types of sources, however the business may demand for integrating Apache Solr with non-J2EE based technology. Although there are various technologies available in market, the placement of Apache Solr like enterprise search engine in broader picture of organization’s IT topology is always targeted for specific areas. This area is nothing but the end user web portals.

Continue Reading

Site Footer

@ OpenSooq 2019