MongoDB DataOps: Fork, Lookup , mongostat, Custom Replication and Taking Care of Failed System

Why fork is used?

fork is the right way to make sure your MongoDB runs as a daemon. In this case, fork should be defined as true in the configuration file. If you want to run it as an interactive process. just set it to false.

How to Perform Lookup?

db.classes.aggregate( [
{ $lookup: {
  from: "members",
  localField: "enrollmentlist",
  foreignField: "name",
  as: "enrollee_info"
}}
])

mongostat and mongotop

These two great tools are provided as part of the MongoDB installation and can be used to detect the queries types and collections that most heavily affect the MongoDB performance.

Simple Backup and Restore

mongodump --out /tmp/dump/
mongorestore mongodb://127.0.0.1 /tmp/dump/

How to Reset Your MongoDB Instance?

Stop the service, remove all content from the /var/lib/mongodb directory, and start the service again:

sudo service mongod stop

sudo rm -rf /var/lib/mongodb

sudo service mongod start

Setting Custom Replication Source

MongoDB replication automatically balances between replication from the Primary (most updated values) and secondaries w/ a low networking latency to minimize WAN network usage and optimize network resources.

However, your can control this behavior and configure specific source rs.syncFrom() command. Please notice that this command result is temporary and will reset to default once mongod is restarted, connection closed between nodes or 30 sec timeout is reached.

Initiate Your Replicaset:

1. Set the replicaset name in the mongod.conf file

2. Change the bindIP to 0.0.0.0 in the config file

3. Run on the first node the rs.initiate command

rs.initiate(

   {

      _id: "rs",

      version: 1,

      members: [

         { _id: 0, host : "172.30.0.158:27017" },

         { _id: 1, host : "172.30.0.151:27017" },

         { _id: 2, host : "172.30.0.219:27017" }

      ]

   }

)

Search for Delta in the Oplog

db.oplog.rs.find({"ts":{$gt : Timestamp(1640074543, 1)}})

Upgrade from MySQL 5.6 to 8.0

Verify your system configuration to select the right package and method for installation

uname -r
> 2.6.32-696.3.2.el6.x86_64  ==> 64bit

uname -r
> Description:    CentOS release 6.9 (Final) => CentOS 6.9

Upgrade to the latest 5.6

Look for the latest MySQL 5.6 minor version and get the package:

wget https://downloads.mysql.com/archives/get/p/23/file/MySQL-5.6.51-1.el6.x86_64.rpm-bundle.tar

Remove current package

sudo service mysqld stop

sudo yum -y remove mysql-community-common-5.6.36-2.el6.x86_64

tar -xvf MySQL-5.6.51-1.el6.x86_64.rpm-bundle.tar

sudo rpm -i MySQL-client-5.6.51-1.el6.x86_64.rpm

sudo rpm -i  MySQL-server-5.6.51-1.el6.x86_64.rpm

sudo service mysql start

sudo mysql_upgrade

Upgrade to the latest 5.7

Look for latest MySQL 5.7 minor version

wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.35-1.el6.x86_64.rpm-bundle.tar

sudo service mysql stop

sudo yum -y remove MySQL-server-5.6.51-1.el6.x86_64 MySQL-client-5.6.51-1.el6.x86_64

tar -xvf mysql-5.7.35-1.el6.x86_64.rpm-bundle.tar

sudo rpm -i mysql-community-libs-5.7.35-1.el6.x86_64.rpm mysql-community-common-5.7.35-1.el6.x86_64.rpm mysql-community-client-5.7.35-1.el6.x86_64.rpm mysql-community-server-5.7.35-1.el6.x86_64.rpm

sudo service mysqld start

sudo mysql_upgrade

Upgrade to the latest 8.0

Look for latest MySQL 8.0 minor version

wget https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.26-1.el6.x86_64.rpm-bundle.tar

sudo service mysqld stop

sudo yum -y remove mysql-community-libs-5.7.35-1.el6.x86_64 mysql-community-common-5.7.35-1.el6.x86_64 mysql-community-client-5.7.35-1.el6.x86_64 mysql-community-server-5.7.35-1.el6.x86_64

tar -xvf mysql-8.0.26-1.el6.x86_64.rpm-bundle.tar

sudo rpm -i mysql-community-client-plugins-8.0.26-1.el6.x86_64.rpm mysql-community-common-8.0.26-1.el6.x86_64.rpm mysql-community-libs-8.0.26-1.el6.x86_64.rpm mysql-community-client-8.0.26-1.el6.x86_64.rpm mysql-community-server-8.0.26-1.el6.x86_64.rpm

sudo service mysqld start

Comparing SQL to MongoDB Query Syntax

#MongoDB Aggregation Pipeline

db.documents.aggregate([
{"$match": {"name": "/^asdasd/"}},
{"$group": {"_id": "$name", {"cnt": {$sum:1}}},
{"$match": {"cnt": {$gt: 3}}},
{"$lookup": {from: "meta", localField:"name", foreignField:"name", as: "meta"}}
{"$match": {"meta": {$exists: True}}},
{"$out": "debug_collection"}
]);

#SQL Syntax

SELECT *
FROM documents

SELECT *
FROM documents
WHERE name like 'asdasd%'

SELECT name, COUNT(*) AS cnt
FROM documents
WHERE name like 'asdasd%'
GROUP BY name

SELECT name, COUNT(*)
FROM documents
WHERE name like 'asdasd%'
GROUP BY name
HAVING COUNT(*) > 3

SELECT doc.*, meta.*
FROM meta INNER JOIN (
SELECT name, COUNT(*)
FROM documents
WHERE name like 'asdasd%'
GROUP BY name
HAVING COUNT(*) > 3) AS doc ON doc.name = meta.name

Your First MongoDB, C# and .NET Core App/My C# and MongoDB Cheat Sheet

Download .NET core SDK

Setup you project and your console project

1. Select a folder
2. In the command line: 
   dotnet new console -o console
   dotnet add console package MongoDB.Driver
   cd console
   code console
3. vsoode will be opened

Your First Basic Program

Create a program.cs file:

namespace HelloWorld

{

    class Program

    {

        static void Main(string[] args)

        {

            Console.WriteLine("Hello World!");

        }

    }

}

Run the code:

dotnet run

Now add a document and write it

using MongoDB.Driver;

using MongoDB.Bson;

using MongoDB.Bson.Serialization.Attributes;

namespace MongoData

{

    class Program

    {

        static void Main(string[] args)

        {

            Console.WriteLine("Hello World!");

            var mongoClient = new MongoClient("MONGODB_URL");

            var mongoDatabase = mongoClient.GetDatabase("trial_db");

            var collection = mongoDatabase.GetCollection<TrialDoc>("trial_cl");

            TrialDoc trialDoc = new TrialDoc();

            trialDoc.BookName = "123";

            collection.InsertOne(trialDoc);

            Console.WriteLine("Done!");

        }

    }

   

    public class TrialDoc

    {

        [BsonId]

        [BsonRepresentation(BsonType.ObjectId)]

        public string? Id { get; set; }

        [BsonElement("Name")]

        public string BookName { get; set; } = null!;

        public int Price { get; set; }

        public string Category { get; set; } = null!;

        public string Author { get; set; } = null!;

    }

}

Add Read Operation: Find All

            var list = collection.Find(_ => true).ToList();

            foreach (var item in list) {

                Console.WriteLine(item.BookName);

            }

Add Read Operation: Specific Record

            var single = collection.Find(x => x.BookName == "123".ToString()).FirstOrDefault();

            Console.WriteLine(single.Id);

Update the record

            collection.ReplaceOne(x => x.Id == single.Id, new TrialDoc() {Id = single.Id, BookName = "456"}, new ReplaceOptions {IsUpsert = true});

Smarter update using findAndModify

var _client = new MongoClient();
var _database = _client.GetDatabase("users");
var counters = _database.GetCollection<BsonDocument>("counters");
var counterQuery = Builders<BsonDocument>.Filter.Eq("_id", "eventId");

var findAndModifyResult = counters.FindOneAndUpdate(counterQuery,
              Builders<BsonDocument>.Update.Set("web", "testweb"));

Aggregation Framework

            FilterDefinition<TrialDoc> filter =

                Builders<TrialDoc>.Filter.Eq(x => x.BookName, "123");

            ProjectionDefinition<TrialDoc> project = Builders<TrialDoc>

                .Projection.Include(x => x.BookName);

            var results = collection.Find(filter).Project(project).ToList();

            foreach (var item in results) {

                Console.WriteLine(item);

            }

Full Text Search

Don't forget to create a full text search index first

            var results_list = collection.Find(Builders<TrialDoc>.Filter.Text("123")).ToList();

            foreach (var item in results_list) {

                Console.WriteLine(item);

            }

GeoSpatial Stuff

using MongoDB.Driver.GeoJsonObjectModel;

#Add to document
public GeoJson2DCoordinates Location { get; set; } = null!;
trialDoc.Location = new GeoJson2DCoordinates(31.11, 19.12);

var keys = Builders<BsonDocument>.IndexKeys.Geo2DSphere("Location");
await collection.Indexes.CreateOneAsync(keys);