sandeepk

In Python, we can define private variables and methods. These variables and methods cannot be accessed out of the class or by the inherited class according to the Object-Oriented Paradigm. Things are a bit different in Python.

In python, we can achieve this by using Name Mangling which means adding __ before the variable or method name, let see how

class PPrint:
    def __init__(self):
        self.__secret_msg = 'I am alive' # This is a private variable
        self.check = False

    def __private_method(self): # This is a private method
        print('I am private method !')
                
    def public_method(self): # This public method, which can access the private method
        self.__private_method()
        
    def public_print(self):
        print(self.__secret_msg)
        print(self.check)
 

In the above class, we define the private methods and variables. Let see how to use them


>>> p = PPrint()
>>> p.public_print()
I am alive
False
>>> p.public_method()
I am private method !
# so far so good, let see what happends when we try to access these private methods or variables.
>>> p.__private_msg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'PPrint' object has no attribute '__private_msg'
>>> p.__private_method()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'PPrint' object has no attribute '__private_method'

As you can see python does not allow access to these private variables and methods. But, If I say you can access these private variables and methods.

>>> dir(p)
['_PPrint__private_method', '_PPrint__secret_msg', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'check', 'public_method', 'public_print']

# You can see we have private method and variable '_PPrint__private_method', '_PPrint__secret_msg'

>>> p._PPrint__private_method()
I am private method !

>>> p._PPrint__secret_msg
'I am alive'

So, that's what Name Mangling do it add *<classname>_<method/variablename>.

Conclusion In Python, we can use the name mangling to define the private method and variables, but it not as strict as in other OOP languages. It's almost like a convention which is followed by all python developer. You may also found _ added before the method and variable name, which is just another way to say don't access these methods directly.

Cheers!

#100DaysToOffload #Python

Let get straight to the expression False == False in [True], what you think this expression evaluates to True or False. Fire up your terminal and try, if you get the answer that you have thought of, congratulations!. My guess was wrong :p. Let break down the expression to see how it is evaluated in Python.

# False == False in [True] 
>>> False == False
True
>>> False in [True]
False
>>> (False == False) in [True]
True
>>>False == (False in [True])
True
# but the weird part is when we run
>>> False == False in [True]
False

Isn't this a bit weird? But it is not so, When we look at the expression the way python interprets the expression, things start making sense.

>>> False == False and False in [True]
False

On a bit further looking. I parse the expression to get the Abstract Syntax Tree, which tell a lot about the expression evaluation.

>>> import ast, pprint
>>> pprint.pprint(ast.dump(ast.parse('False == False in [True]')), indent=4)
('Module(body=[Expr(value=Compare(left=Constant(value=False, kind=None), '
 'ops=[Eq(), In()], comparators=[Constant(value=False, kind=None), '
 'List(elts=[Constant(value=True, kind=None)], ctx=Load())]))], '
 'type_ignores=[])')

From the above output of AST you can see that [Eq(), In()] are compound operators and in python, the precedence of these operations is the same.

comparisons, including tests, which all have the same precedence chain from left to right

The below mentioned operators have the same precedence.

in, not in, is, is not, <, <=, >, >=, <>, !=, ==

So, when Python tries to evaluate the expression False == False in [True], it encounters the operators is and == which have the same precedence, so it performs chaining from left to right. – from stackoverflow[0]

# so above expression on evaluating from left to right is done like this.
>>> False == False and False in [True]

Cheers!

References – [0] https://stackoverflow.com/questions/31354429/why-is-true-is-false-false-false-in-python – [1] https://stackoverflow.com/questions/12658197/what-is-the-operator-precedence-when-writing-a-double-inequality-in-python-expl – [2] https://www.youtube.com/watch?v=mRPU3l54Z7I&list=PLWBKAf81pmOamJfoHz4oRdieWQysmUkaW

#100DaysToOffload #Python

These are the terms which you have heard a lot about nowadays in the software world. This is my effort to explain these terms simply. Why they can be of great help to you.

Virtual Machines as Wikipedia states

In computing, a virtual machine (VM) is the virtualization/emulation of a computer system. Virtual machines are based on computer architectures and provide the functionality of a physical computer. Their implementations may involve specialized hardware, software, or a combination.

Let me put that easily, virtual machines are the copy/ soft copy of a computer system which you can run on top of the computer which runs fully independent of the original system. Each virtual machine provides its virtual hardware, including CPUs, memory, hard drives, network interfaces.

Virtual Machines are mainly used for testing, creating operating system backups, and running software or applications on the operating system.

Docker as Wikipedia states

Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.Because all of the containers share the services of a single operating system kernel, they use fewer resources than virtual machines.

Let me put that in a simple way, Docker is the tool to package the software or application in a container that is lightweight and can run on the different operating system, that said, but the kernel needs to be the same in this case. Windows container can run on Windows only and Linux container can run on Linux only.

So, in the above paragraph, we have used the term Image and Container. Let understand what are they,

Image as in real life is a snapshot of the things at a particular time interval. In software, images are the immutable(cannot be changed) snapshot of the application. Image is the collection of libraries and other configuration bundle together.

Container, like the real container, is the box which provide the safe environment for the image to run as an isolated process.

So, Docker comes into the picture to package these image and run this container on top of the machines. Docker has three essential things.

  • Dockerfile
  • Image
  • Container

Conclusion Virtual Machine and Docker are both very useful tools depending on the use case. Virtual machines are very much helpful in case of testing and running software which you do not want to interact with your real system and on the other hand, Docker easy out the deployment process and sharing your application with other people.

Cheers!

#100DaysToOffload #Docker #VirtualMachines

Django provides the support of using multiple databases in your project. Let's see how we can do that, but first, let me put some use case where we might need multiple databases for our application. Why need multiple databases? In today's world, we are gathering a lot of data from user which is used for different purposes, some data is relational data and other is non-relational data. Let me put few use cases

  • Suppose you need to record all the touchpoints of a web page in your web application, for this you need a non-relation database to store that data to run some analytical result on it.
  • Read replicas, you need to set up read replicas of your default database to speed up the fetching of data from the database.
  • Saving the email metadata like how many emails were sent, open rate, error rate, link clicked to see the engagement of the emails.

Lets us see how to set up multiple databases in the Django project.

  1. Need to add the details of the databases in settings.py of Django project.
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.mysql",
        "NAME": config.get("database_default", "name"),
        "USER": config.get("database_default", "user"),
        "PASSWORD": config.get("database_default", "password"),
        "HOST": config.get("database_default", "host"),
        "PORT": "3306",
        "CONN_MAX_AGE": 0,
    },
    "replica1": {
        "ENGINE": "django.db.backends.mysql",
        "NAME": config.get("database_replica1", "name"),
        "USER": config.get("database_replica1", "user"),
        "PASSWORD": config.get("database_replica1", "password"),
        "HOST": config.get("database_replica1", "host"),
        "PORT": "3306",
        "CONN_MAX_AGE": 0,
    },
    "mongo": {
        "ENGINE": "djongo",
        "NAME": config.get("mongo_database", "name"),
        "HOST": config.get("mongo_database", "host"),
        "USER": config.get("mongo_database", "user"),
        "PASSWORD": config.get("mongo_database", "password"),
    },
}

Here you can see we define 2 databases other than the default databases mongo and replica1. After this, you need to tell the Django router in which app you want to use which connection of database. This is one of the ways to do it, you can manually decide which database you want to use while querying.

DATABASE_ROUTERS = ['path.to.replica1', 'path.to.mongo']
  1. Now we need to define this router class to tell them which database to use, for that we need to write a class
class MongoRouter:
    """
    A router to control all database operations on models in the
    analytics and status applications.
    """
    route_app_labels = {'analytics', 'status'}

    def db_for_read(self, model, **hints):
        """
        Attempts to read analytics and status models go to mongo db.
        """
        if model._meta.app_label in self.route_app_labels:
            return 'mongo'
        return None

    def db_for_write(self, model, **hints):
        """
        Attempts to write analytics and status models go to auth_db.
        """
        if model._meta.app_label in self.route_app_labels:
            return 'mongo'
        return None

    def allow_relation(self, obj1, obj2, **hints):
        """
        Allow relations if a model in the analytics and status apps is
        involved.
        """
        if (
            obj1._meta.app_label in self.route_app_labels or
            obj2._meta.app_label in self.route_app_labels
        ):
           return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Make sure the analytics and status apps only appear in the
        'mongo' database.
        """
        if app_label in self.route_app_labels:
            return db == 'mongo
        return None

similar goes for replica1 database


class ReplicaRouter:
    def db_for_read(self, model, **hints):
        """
        Reads go to replica1.
        """
        return 'replica1'

    def db_for_write(self, model, **hints):
        """
        Writes always go to default.
        """
        return 'default'

    def allow_relation(self, obj1, obj2, **hints):
        """
        Relations between objects are allowed if both objects are
        in the default/replica1 pool.
        """
        db_set = {'default', 'replica1'}
        if obj1._state.db in db_set and obj2._state.db in db_set:
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        All non-mongo models end up in this pool.
        """
        return True

That's it, now you read to use multiples database in your project, which we early handle by the routers class you have defined.

Cheers!

#100DaysToOffload #Django #Python

In Django, we can use the abstraction concept in defining the tables for the columns which are common. We can make any modal as an abstract model by adding this meta property abstract = true.

Suppose you have some column fields which are common in all the tables, which you can abstract and have to just inherit this abstract class to add the fields in the model which can help you to follow the Don't Repeat Yourself principle. Let see an example

class Base(models.Model):
  """
  Base parent class for all the models
  """
  timestamp = models.DateTimeField(blank=True, db_index=True)
  is_active = models.BooleanField(default=True, db_index=True)

  def __init__(self, *args, **kwargs):
    super(Base, self).__init__(*args, **kwargs)

  class Meta:
    abstract = True

class OttPlatform(Base):
  """
  """

  name = models.CharField(max_length=200)
  ott_type = models.CharField(max_length=50)

  def __str__(self):
    return self.name

So, this helps you to stop duplication of code, but there is one more issue we can handle here. The is_active column is used to mark the row as deleted. Mainly in our use case, we can't delete the data from the table to keep the track of changes. So is_active field helps us with that. But now we have to use the is_active filter in every query.

We can solve this by overriding the manager, let see how


# First, define the Manager subclass.
class AtiveOTTManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(is_active=True)

class OttPlatform(Base):
  """
  """

  name = models.CharField(max_length=200)
  ott_type = models.CharField(max_length=50)
  
   # the order matters, first come default manager, then custom managers.
  objects = models.Manager() # The default manager.
  active_objects = AtiveOTTManager() # The active OTT manager.

  def __str__(self):
    return self.name

# Now you have to do OttPlatform.active_objects.all(), to get all the active OTT platform name.

So, with overriding the manager we don't have to write a filter for is_active in every query.

Cheers!

#100DaysToOffload #django #python

Design by Contract is the software designing process that takes a contract based approach in developing the software that does no more and no less than it claims to do.

What is Contract? A contract is a document that defines the rights and responsibilities of both the party involved in the agreement and which list the repercussions if either party fails to abide by the contract. We all have seen or been in contact with your Landlord or the employment contract that specifies the roles and responsibilities that you must fulfill.

A Similar process we follow while developing the software, where we focus on documenting (and agreeing to) the rights and responsibilities of software modules to ensure program correctness. While writing the contract these questions will be helpful to get things clear.

  • What does the contract expect?
  • What does the contract guarantee?
  • What does the contract maintain?

The software which follows the Design by Contract has these condition specified

  • Preconditions, are the condition which must be true before calling the routine, if violated routine should never be called.
  • Postconditions, after the routines are finished the state which needs to be achieved should be achieved.
  • Class Invariant, A class ensures that this condition is always true from the perspective of a caller. During internal processing of a routine.

Why Design by Contract can be a good approach ?

  • DBC doesn’t require any setup or mocking
  • DBC we can define both the success and failure cases.
  • DBC can be used during the design phase, development, and deployment phase.
  • DBC fits in nicely with our concept of crashing early.

Conclusion Most of you are thinking, do we need another development approach as we already have Test-driven development (TDD). DBC and TDD are different approaches to the broader topic of the software development process. They both have value and both have used in different situations. DBC approach can be used across the Design, Development, and Deployment. DBC fits perfectly in the world where we follow the concept of crashing early. So give it a shot, will be happy to discuss it, you can reach me out her

Cheers!

#100DaysToOffload #SoftwareDevelopment #DBC #DesignByContract

Django Q() object helps to define SQL condition on the database and can be combined with the &(AND) and |(or) operator. Q() helps in the flexibility of defining and reusing the conditions.

  • Using Q() objects to make an AND conditions.
  • Using Q() objects to make an OR conditions.
  • Using Q() objects to make reusable conditions.

Using Q() objects to make an AND conditions We can use Q() objects to combine multiple filter conditions into one condition as filter conditions always perform AND operations.

from django.db.models import Q

# Without Q() object
document_obj = Document.objects.filter(created_by=1282).filter(doc_type='purchase_order').filter(edit=0).filter(cancelled=0)

#With Q() object
q_filter_document = Q(created_by=1282) & Q(doc_type='purchase_order') & Q(cancelled=0) &(edit=0)

# can also be written as
q_filter_document_another_way = Q(created_by=1282, doc_type='purchase_order', cancelled=0, edit=0)

document_obj = Document.objects.filter(q_filter_document)

Using Q() objects to make an OR conditions

from django.db.models import Q


#With Q() object
q_filter_document = Q(created_by=1282) | Q(created_by=1282)
document_obj = Document.objects.filter(q_filter_document)

Q() to make reusable filter condition The best use of Q() objects is reusability, we define the Q() once and can use them to combine with different Q() objects with help of &, |, and ~ operators.

Let's consider a use case, in which the user can generate a report based on certain filters. User can filter report based on these values documenttype, isdraft, createdby, documentstatus


def get_document_object(document_type, is_draft, created_by, document_status):
    base_query = Q(active=1, cancelled=0, document_tye=document_type, is_draft=is_draft)

    # based on condition we can different Q() objects to filter the tables
    if document_status = 'in_progress':
        base_query = base_query & Q(document_status=document_status, completed=0)
    else if document_status = 'completed':
        base_query = base_query & Q(document_status=document_status, completed=1)


   return Documents.objects.filter(base_query)

In Q() objects we can use the same conditional operator which we use in filter objects like in operator, startswith, endswith, etc.

Conclusion Q() objects contribute to clean code and reusability. It helps to define the condition with &, |, and ~ relation operator to simplify the complex queries.

Cheers!

#django #python #100DaysToOffload

In the previous blog post, we have discussed F() Expression, we will now explore more query expression in Django, to name few that we will discuss in this post are

  • Func() Expression
  • Subquery Expression
  • Aggregation () Expression

Func() Expression Func () Expression is the base of all the expressions and can be used to create your custom expression for the database level function.

# The table that we using for our query is the *Student* which keeps records of the students for the whole school.

from django.db.models import F, Func
student_obj = Student.objects.annotate(full_name=Func(F('first_name') + F('last_name'), function='UPPER')

# This will give a student object with a new field that is *full_name* of the student in upper case.

Subquery Expression Subquery are like nested condition in the query filter which helps you to make a complex query into a clean concise query. But you need to know the order of the sequence the query will be executed to use effectively. While using a Subquery you will also need to know about the OuterRef, which is like an F() Expression but points to the parent query value, let see both Subquery and OuterRef in action

# you are given a task to get the name of the student whose name starts with *S* and whose fees are due.

from django.db.models import OuterRef, Subquery
fee_objects = Fees.objects.filter(payment_due_gt=0)
student_obj = Student.objects.filter(name__startswith='S').filter(id__in=Subquery(fee_objects.values('student_id')))

# Get the lastest remarks for the students
remark = Remark.objects.filter(student_id=OuterRef('pk')).order_by('-created_at')
student_obj = Student.objects.annotate(newest_remark=Subquery(remark.values('remark_strl')[:1]))

Aggregation () Expression

Aggregation Expression is the Func Expression with GroupBy clause in the query filter.

# get the total student enrolled in the *Blind Faith* subject.

student_obj = Student.objects.filter(subject_name='blind_faith').annotate(total_count=Count('id'))

Note: All queries mentioned above in the code are not tested. So if you see any typo, a query that does not make sense, feel free to reach out to me at sandeepchoudhary1507[at]gmail[DOT]com.

Cheers!

#100DaysToOffload #django #python

While working from home one of the issue I faced that my laptop battery charger adapter is always remain plugged in almost all the time, due to which I have to replace my laptop battery. To deal with the problem I have now written a script to notify me about the laptop battery charging level if it goes above 85 % and below 20%.

#! /bin/bash                                                                                                                                          

while true
do

    battery_level=`acpi -b | grep -o '[0-9]*' | sed -n  2p`
    ac_power=`cat /sys/class/power_supply/AC/online`

    #If above command raise an error " No such file or directory" try the below command.
    #ac_power=`cat /sys/class/power_supply/ACAD/online`
    if [ $ac_power -gt 0 ]; then
        if [ $battery_level -ge 85 ]; then
            notify-send "Battery Full" "Level: ${battery_level}%"
        fi
    else
        if [ $battery_level -le 20 ]; then
            notify-send --urgency=CRITICAL "Battery Low" "Level: ${battery_level}%"
        fi
    fi
    sleep 120

done

The important commands which I want to break down to explain actually what they are doing.

  • First is acpi which tell us about the battery information and other ACPI information
  • grep command is used to extract the integer value from the acpi command
  • sed is used to get the second value from the grep result.
>> acpi -b
Battery 0: Discharging, 54%, 02:03:37 remaining
>>acpi -b | grep -o '[0-9]*'
0
54
02
04
41
>>acpi -b | grep -o '[0-9]*' | sed -n  2p
54
  • After that, we check that the charger is plugged in or not, based on that we check that the battery level does not exceed the described limit, if so is the case send the notification.
  • Then we check for the battery low indication which sends the notification if the battery level less than 20 %.
  • These condition is put in a continuous loop to check after a sleep time of 120s.

To make this script run automatically you have to assign the execution permission and specifies the execution command in the ~/.profile and reboot the system.

>> sudo chmod +x /path/to/battery-notification.sh

you can find my notes on shell commands here

thanks shrini for pointing out issue for Ubuntu 20 in lineac_power=`cat /sys/class/power_supply/AC/online` :) Cheers!

#100DaysToOffload #automation #scripts

What is the F() Expression? First let me explain to you what are Query Expressions are, these expressions let you use value or computation to be used in the update, create and filters, order by, annotation, aggregation. F() object represent the value of the model fields or annotated columns. It lets you help to not load the value of the field into the python memory rather directly handles in the Database query.

How to use the F() Expression? To use the F expression you have to import them from the from django.db.models import F and have to pass the name of the field or annotated column as argument, and it will return the value of the field from the database, without letting know the python any value. Let some example.

from django.db.models import F

# Documents is the table which have the details of the document submitted by user from the registrey portal for GYM membership

# We need update the count of the document submitted by the user with pk=10091

# without using F Expression

document = Documents.objects.get(user_id=10091)
document.document_counts += 1
document.save()

# Using F expression
document = Documents.objects.get(user_id=10091)
document.document_counts = F('document_counts') + 1
document.save()

Benefits of the F() Expression.

  • With the help of F expression we can make are query clean and concise.
    from django.db.models import F
  document = Documents.objects.get(user_id=10091)
  document.update(document_counts=F('document_counts') + 1)

   #Here we also have achieved some performance advantage
    #1. All the work is done at database level, rather than throwing the value from the database in the python memory to do the computation.
    #2. Save queries hit on the database.
  • F Expression can save you from the race condition. Consider a scenario where multiple user access your database and when bother user access the Document object for the user 10091, the count value is two, when user updates the value and save it and other user does the same the value will be saved as three not Four because when both user fetches the value its two.

  # user A fetch the document object, and value of document_counts is two.
  document = Documents.objects.get(user_id=10091)
  document.document_counts += 1
  document.save()
  # after the operation value of document_counts is three

  # Code running prallerly, User B also fetch the object, and value of document_counts is two.
  document = Documents.objects.get(user_id=10091)
  document.document_counts  += 1
  document.save()
  # after the operation value of document_counts is three

  # But actually value should be Four, but this is not the case using F expression here will save use from this race condition.
  • F Expression are persistent, which means the query persist after the save operation, so you have to use the refreshfromdb to avoid the persistence.
  document = Documents.objects.get(user_id=10091)
  document.document_counts = F('document_counts') + 1
  document.save()

  document.document_validation = 0
  document.save()

  # This will increase the value of *document_counts* by two rather then one as the query presists and calling save will trigger the increment again.

  • More example of F Expression in action with filter, annotate query.
from django.db.models import F

# annotation example
annotate_document = Document.objects.filter(created_by=F('created_by_first_name') + F('created_by_last_name')


# filter example
filter_document = Documents.objects.filter(total_documents_count__gt=F('valid_documents_count'))

That's all about the F Expression, it is pretty handy and helps you improve the performance, by reducing value loading in the memory and query hit optimization, but you have to keep an eye over the persistent issue, which I assume will not be the case if you are writing your code in structured way.

Cheers!

#100DaysToOffload #django #python