sandeepk

Today we are gone to talk about the Foreign Key which is used to establish a many-to-one relationship and different args which can be passed.

Syntax to define a Foreign key relationship

from django.db import models

class GSTRate(models.Model):
    # ...
    pass

class PurchaseOrder(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE)

Different args for the Foreign Key

  • on_delete
  • limitchoicesto
  • related_name
  • relatedqueryname
  • to_field
  • db_constraints

on_delete This args handle the process, of what to do if the reference keys are deleted, we have three options – CASCADE: It deletes the objects contains the Foreign key – PROTECT: It prevents the deletion of the referenced object by raising ProtectedError – RESTRICT: Introduced in Django 3.1, It only deletes the object, if another references object is being deleted in the same operation, but with CASCADE flow otherwise raise RestrictedError

from django.db import models

class GSTRate(models.Model):
    # ...
    pass

class PurchaseOrder(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE)

limit_choices_to This is helpful in the form rendering to restrict the option for the reference field. The limited choice can either be a dict, Q object or a callable that return dict or Q object

from django.db import models

class GSTRate(models.Model):
    # ...
    pass

class PurchaseOrder(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE, limit_choices_to={'tax_type': 'gst'},)
# here we only show the tax of type GST only.

related_name It's the name that can be used for the reverse or backward reference while querying. You can also stop the backward relation by assigning + in related_name.


class GSTRate(models.Model):
    # ...
    pass

class PurchaseOrder(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE, related_name='+',) # IN this case GSTRate cannot backward realte to the Purchase Order obejct.

class PurchaseOrder(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE, related_name='invoice')

# here you can refer to Invoice inside GSTRate query by the related name.
GSTRate.objects.filter(invoice_id=1232)

related_query_name It has the same value as the related_name, unless not specified explicitly. If it is defined then you have to use that name while querying


class GSTRate(models.Model):
    # ...
    pass

class OrderConfirmation(models.Model):
    gst = models.ForeignKey(GSTRate, on_delete=models.CASCADE, related_name='order_confirmation', related_query_name='oc')

# here you can refer to Invoice inside GSTRate query by the related name.
GSTRate.objects.filter(oc_id=112)

to_field This is helpful in the case where you want to refer other the primary key of the modal, but the condition whatever key you referred it should be unique=True

db_constraints This is used to create a constraint at the database level, by default value is true and if the user sets the value to False, then accessing a related object that doesn’t exist will raise its DoesNotExist exception.

Conclusion These args are handy which gives us the possibilities of implementing the additional constraint and functionality to query data in the Django ORM query language.

Cheers!

#100DaysToOffload #Python#Django

I have started my financial journaling a few months back, and I am happy with the simplicity and freedom that I can have with the use of Emacs, ledger-mode, and Ledger. I have written about it before, you can check that here. Today we will talk about report generation.

So, for this post, we will be using this mock data for generating balance and monthly expense reports. Create a new file with the name ledger.dat and paste the data in it given below.

2021/02/14 * Dinner-Party
    Expenses:Food:Party                            1500
    Assets:Bank:Checking

2021/02/21 * Lunch-Party
    Expenses:Food:Party                           1600
    Assets:Bank:Checking

2021/02/22 * Home
    Expenses:Home                                   3000
    Assets:Bank:Checking                       3000
    Assets:Investment:Stocks                 180000
    Assets:Bank:Checking

2021/02/25 * Amazon Shopping
    Expenses:Self                                60000
    Assets:Bank:Checking

So let's start with the balance report, open the Emacs with the ledger.dat file, then press C-c C-o C-r. The buffer will ask the report name type in bal and hit enter, you will get the report.

Report: bal
Command: ledger -f /home/ledger.dat bal

              -66100  Assets
             -246100    Bank:Checking
              180000    Investment:Stocks
	       66100  Expenses
		3100    Food:Party
		3000    Home
	       60000    Self
--------------------
                   0

There is another way which I use mainly is from the terminal itself.

>>>  ledger balance -f ledger.dat
              -66100  Assets
             -246100    Bank:Checking
              180000    Investment:Stocks
               66100  Expenses
                3100    Food:Party
                3000    Home
               60000    Self
--------------------
                   0

If you only want to see a particular type of expense, you can easily do so by

# here we are only seeing the expenses
>>> ledger balance -f ledger.dat Expenses
               66100  Expenses
                3100    Food:Party
                3000    Home
               60000    Self
--------------------
               66100

# here we are seeing the Checking account
>>>ledger balance -f dummy.dat Checking
             -246100  Assets:Bank:Checking

We can also see month-wise expense, for that, I have changed the date in the file ledger.dat.

>>> ledger -M --period-sort "(amount)" reg ^expenses -f ledger.dat
21-Jan-01 - 21-Jan-31                             Expenses:Food:Party                                              1600                    1600
21-Feb-01 - 21-Feb-28                             Expenses:Food:Party                                              1500                    3100
                                                  Expenses:Self                                                   60000                   63100
21-Mar-01 - 21-Mar-31                             Expenses:Home                                                    3000                   66100

Reports help to see your money flow, calculate the expenses and savings.

Cheers!

#100DaysToOffLoad #Emacs #LedgerMode #Ledger

I have written this script a few months back, that runs the application that I use daily for work, If I login into the system at a specific time interval :)

In this logic we check what is the day of the week is, If it is Saturday or Sunday exit the script. The command used here is – cut – remove sections from each line of files – d – use DELIM instead of TAB for the field delimiter – f select only these fields; also print any line that contains no delimiter character, unless the -s option is specified*

day=$(date | cut -d " " -f1)
not_days=(Sun Sat)
if [[ " ${not_days[@]} " =~ $day ]]
then
    exit 1
fi

Here we're calculating the difference with the current time on the system, If the difference is less than one, we will specify the application we want to run, else exit.

dbtimestamp=1050
secondsDiff=$(( `date '+%H%M'` - $dbtimestamp ))
echo $secondsDiff

So here is the full script

#!/bin/bash                                                               

day=$(date | cut -d " " -f1)
not_days=(Sun Sat)
if [[ " ${not_days[@]} " =~ $day ]]
then
    exit 1
fi

dbtimestamp=1050
secondsDiff=$(( `date '+%H%M'` - $dbtimestamp ))

if [ $secondsDiff -lt 1 ]
then
  hexchat &
  firefox &
  safeeyes &
 # other application you want to open
else
  exit 0
fi ;

There are some improvements which I will do in the scripts in future – Time delta logic for running the application needs to be improved. – Flag to override the all check condition and run the applications

Cheers!

#100DaysToOffload #Automation #Bash

Template is the one of powerful features of Django Framework, and we will discuss the tag and filter. If you do not have any idea about the template, please go read about that here, I will wait for you.

Template tags which we are going to discuss today are

  • URL
  • Now
  • Spaceless

URL The URL tag is the template equivalent of the reverse function. URL can accept args or kwargs for routes that expect other variables.

<a href="{% url "a_named_view" %}">Go to a named view</a>

Now Now is a convenient method to display information about the current time. It was a problem solver for the copyright year update, which I first have to update every year manually.

&copy; {% now "Y" %} sandeepk.dev.

Spaceless It removes the whitespace, newline, tab from the HTML

{% spaceless %}
    <div>
        <span>I am a good span.</span>
    </div>
{% endspaceless %}

Now let's talk about the filter which we can use in the template.

  • date
  • default
  • pluralize
  • yesno

Date This filter can be used to control the format of the datetime passed in the context of the response.

{{ birth_date|date:"Y-m-d" }}

Default Have you ever been in a condition when you want to show a text for a falsy value, then default filter is the one thing you should use

{{ value_to_check|default:"Nothing to see here." }}

Pluralize When your text considers counts of things. Pluralize can use to save you from if and else check

{ count_items }} item{{ count_items|pluralize }}

0 items
1 item
2 items
3 items
(and so on)

YesNo YesNo is good for converting True|False|None into a meaningful text message

{{ user.name }} has {{ user_accepted|yesno:"accepted,declined,not RSVPed" }} request.

These are the tags and filter which I use very often but there many more which you can check out here

Cheers!

#100DaysToOffload #Python #Django

Process To put it simply process is a program in an execution state, which has process id, process state, and priority. The process does not share the memory between each other and is unaware of the other process. States of the Process – Ready – Running – Waiting – Terminated – Suspended.

Thread Thread is the segment of the process which means a process can have multiple threads with shared memory access. A thread can execute any part of the process code, including parts currently being executed by another thread. States of the Thread – Running – Ready – Blocked.

Now we have some understanding of the Process and Thread. Let see the thread and process currently running in our system.

>>> top # show the process

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                           
 3213 sandeepk  20   0 3422616 389668 166132 S  15.5  4.8   3:35.47 firefox                                                                           
 4679 sandeepk  20   0 2931264 284812 144300 S  13.2  3.5   0:27.11 Web Content                                                                       
 1502 root      20   0  667788  87748  58564 S   7.3  1.1   0:37.59 Xorg                                                                       

Now let see the thread of the Process with PID: 3213

>>> ps -T -p 3213
  PID  SPID TTY          TIME CMD
 3213  3213 ?        00:01:45 firefox
 3213  3413 ?        00:00:17 IPC I/O Parent
 3213  3497 ?        00:00:01 Timer
 3213  3498 ?        00:00:00 Netlink Monitor
 3213  3499 ?        00:00:06 Socket Thread
 3213  3540 ?        00:00:00 Permission
 3213  3555 ?        00:00:00 gmain
 3213  3556 ?        00:00:00 gdbus
 3213  3560 ?        00:00:00 JS Watchdog
 3213  3561 ?        00:00:00 JS Helper
 3213  3562 ?        00:00:00 JS Helper
 3213  3563 ?        00:00:00 JS Helper
 3213  3564 ?        00:00:00 JS Helper
 3213  3609 ?        00:00:02 Cache2 I/O
 3213  3610 ?        00:00:00 Cookie
 3213  3640 ?        00:00:00 Worker Launcher
 3213  3678 ?        00:00:05 Softwar~cThread

So, that's all from my side about Thread and Process.

Cheers!

#100DaysToOffload

In Python, we can define private variables and methods. These variables and methods cannot be accessed out of the class or by the inherited class according to the Object-Oriented Paradigm. Things are a bit different in Python.

In python, we can achieve this by using Name Mangling which means adding __ before the variable or method name, let see how

class PPrint:
    def __init__(self):
        self.__secret_msg = 'I am alive' # This is a private variable
        self.check = False

    def __private_method(self): # This is a private method
        print('I am private method !')
                
    def public_method(self): # This public method, which can access the private method
        self.__private_method()
        
    def public_print(self):
        print(self.__secret_msg)
        print(self.check)
 

In the above class, we define the private methods and variables. Let see how to use them


>>> p = PPrint()
>>> p.public_print()
I am alive
False
>>> p.public_method()
I am private method !
# so far so good, let see what happends when we try to access these private methods or variables.
>>> p.__private_msg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'PPrint' object has no attribute '__private_msg'
>>> p.__private_method()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'PPrint' object has no attribute '__private_method'

As you can see python does not allow access to these private variables and methods. But, If I say you can access these private variables and methods.

>>> dir(p)
['_PPrint__private_method', '_PPrint__secret_msg', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'check', 'public_method', 'public_print']

# You can see we have private method and variable '_PPrint__private_method', '_PPrint__secret_msg'

>>> p._PPrint__private_method()
I am private method !

>>> p._PPrint__secret_msg
'I am alive'

So, that's what Name Mangling do it add *<classname>_<method/variablename>.

Conclusion In Python, we can use the name mangling to define the private method and variables, but it not as strict as in other OOP languages. It's almost like a convention which is followed by all python developer. You may also found _ added before the method and variable name, which is just another way to say don't access these methods directly.

Cheers!

#100DaysToOffload #Python

Let get straight to the expression False == False in [True], what you think this expression evaluates to True or False. Fire up your terminal and try, if you get the answer that you have thought of, congratulations!. My guess was wrong :p. Let break down the expression to see how it is evaluated in Python.

# False == False in [True] 
>>> False == False
True
>>> False in [True]
False
>>> (False == False) in [True]
True
>>>False == (False in [True])
True
# but the weird part is when we run
>>> False == False in [True]
False

Isn't this a bit weird? But it is not so, When we look at the expression the way python interprets the expression, things start making sense.

>>> False == False and False in [True]
False

On a bit further looking. I parse the expression to get the Abstract Syntax Tree, which tell a lot about the expression evaluation.

>>> import ast, pprint
>>> pprint.pprint(ast.dump(ast.parse('False == False in [True]')), indent=4)
('Module(body=[Expr(value=Compare(left=Constant(value=False, kind=None), '
 'ops=[Eq(), In()], comparators=[Constant(value=False, kind=None), '
 'List(elts=[Constant(value=True, kind=None)], ctx=Load())]))], '
 'type_ignores=[])')

From the above output of AST you can see that [Eq(), In()] are compound operators and in python, the precedence of these operations is the same.

comparisons, including tests, which all have the same precedence chain from left to right

The below mentioned operators have the same precedence.

in, not in, is, is not, <, <=, >, >=, <>, !=, ==

So, when Python tries to evaluate the expression False == False in [True], it encounters the operators is and == which have the same precedence, so it performs chaining from left to right. – from stackoverflow[0]

# so above expression on evaluating from left to right is done like this.
>>> False == False and False in [True]

Cheers!

References – [0] https://stackoverflow.com/questions/31354429/why-is-true-is-false-false-false-in-python – [1] https://stackoverflow.com/questions/12658197/what-is-the-operator-precedence-when-writing-a-double-inequality-in-python-expl – [2] https://www.youtube.com/watch?v=mRPU3l54Z7I&list=PLWBKAf81pmOamJfoHz4oRdieWQysmUkaW

#100DaysToOffload #Python

These are the terms which you have heard a lot about nowadays in the software world. This is my effort to explain these terms simply. Why they can be of great help to you.

Virtual Machines as Wikipedia states

In computing, a virtual machine (VM) is the virtualization/emulation of a computer system. Virtual machines are based on computer architectures and provide the functionality of a physical computer. Their implementations may involve specialized hardware, software, or a combination.

Let me put that easily, virtual machines are the copy/ soft copy of a computer system which you can run on top of the computer which runs fully independent of the original system. Each virtual machine provides its virtual hardware, including CPUs, memory, hard drives, network interfaces.

Virtual Machines are mainly used for testing, creating operating system backups, and running software or applications on the operating system.

Docker as Wikipedia states

Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.Because all of the containers share the services of a single operating system kernel, they use fewer resources than virtual machines.

Let me put that in a simple way, Docker is the tool to package the software or application in a container that is lightweight and can run on the different operating system, that said, but the kernel needs to be the same in this case. Windows container can run on Windows only and Linux container can run on Linux only.

So, in the above paragraph, we have used the term Image and Container. Let understand what are they,

Image as in real life is a snapshot of the things at a particular time interval. In software, images are the immutable(cannot be changed) snapshot of the application. Image is the collection of libraries and other configuration bundle together.

Container, like the real container, is the box which provide the safe environment for the image to run as an isolated process.

So, Docker comes into the picture to package these image and run this container on top of the machines. Docker has three essential things.

  • Dockerfile
  • Image
  • Container

Conclusion Virtual Machine and Docker are both very useful tools depending on the use case. Virtual machines are very much helpful in case of testing and running software which you do not want to interact with your real system and on the other hand, Docker easy out the deployment process and sharing your application with other people.

Cheers!

#100DaysToOffload #Docker #VirtualMachines

Django provides the support of using multiple databases in your project. Let's see how we can do that, but first, let me put some use case where we might need multiple databases for our application. Why need multiple databases? In today's world, we are gathering a lot of data from user which is used for different purposes, some data is relational data and other is non-relational data. Let me put few use cases

  • Suppose you need to record all the touchpoints of a web page in your web application, for this you need a non-relation database to store that data to run some analytical result on it.
  • Read replicas, you need to set up read replicas of your default database to speed up the fetching of data from the database.
  • Saving the email metadata like how many emails were sent, open rate, error rate, link clicked to see the engagement of the emails.

Lets us see how to set up multiple databases in the Django project.

  1. Need to add the details of the databases in settings.py of Django project.
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.mysql",
        "NAME": config.get("database_default", "name"),
        "USER": config.get("database_default", "user"),
        "PASSWORD": config.get("database_default", "password"),
        "HOST": config.get("database_default", "host"),
        "PORT": "3306",
        "CONN_MAX_AGE": 0,
    },
    "replica1": {
        "ENGINE": "django.db.backends.mysql",
        "NAME": config.get("database_replica1", "name"),
        "USER": config.get("database_replica1", "user"),
        "PASSWORD": config.get("database_replica1", "password"),
        "HOST": config.get("database_replica1", "host"),
        "PORT": "3306",
        "CONN_MAX_AGE": 0,
    },
    "mongo": {
        "ENGINE": "djongo",
        "NAME": config.get("mongo_database", "name"),
        "HOST": config.get("mongo_database", "host"),
        "USER": config.get("mongo_database", "user"),
        "PASSWORD": config.get("mongo_database", "password"),
    },
}

Here you can see we define 2 databases other than the default databases mongo and replica1. After this, you need to tell the Django router in which app you want to use which connection of database. This is one of the ways to do it, you can manually decide which database you want to use while querying.

DATABASE_ROUTERS = ['path.to.replica1', 'path.to.mongo']
  1. Now we need to define this router class to tell them which database to use, for that we need to write a class
class MongoRouter:
    """
    A router to control all database operations on models in the
    analytics and status applications.
    """
    route_app_labels = {'analytics', 'status'}

    def db_for_read(self, model, **hints):
        """
        Attempts to read analytics and status models go to mongo db.
        """
        if model._meta.app_label in self.route_app_labels:
            return 'mongo'
        return None

    def db_for_write(self, model, **hints):
        """
        Attempts to write analytics and status models go to auth_db.
        """
        if model._meta.app_label in self.route_app_labels:
            return 'mongo'
        return None

    def allow_relation(self, obj1, obj2, **hints):
        """
        Allow relations if a model in the analytics and status apps is
        involved.
        """
        if (
            obj1._meta.app_label in self.route_app_labels or
            obj2._meta.app_label in self.route_app_labels
        ):
           return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Make sure the analytics and status apps only appear in the
        'mongo' database.
        """
        if app_label in self.route_app_labels:
            return db == 'mongo
        return None

similar goes for replica1 database


class ReplicaRouter:
    def db_for_read(self, model, **hints):
        """
        Reads go to replica1.
        """
        return 'replica1'

    def db_for_write(self, model, **hints):
        """
        Writes always go to default.
        """
        return 'default'

    def allow_relation(self, obj1, obj2, **hints):
        """
        Relations between objects are allowed if both objects are
        in the default/replica1 pool.
        """
        db_set = {'default', 'replica1'}
        if obj1._state.db in db_set and obj2._state.db in db_set:
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        All non-mongo models end up in this pool.
        """
        return True

That's it, now you read to use multiples database in your project, which we early handle by the routers class you have defined.

Cheers!

#100DaysToOffload #Django #Python

In Django, we can use the abstraction concept in defining the tables for the columns which are common. We can make any modal as an abstract model by adding this meta property abstract = true.

Suppose you have some column fields which are common in all the tables, which you can abstract and have to just inherit this abstract class to add the fields in the model which can help you to follow the Don't Repeat Yourself principle. Let see an example

class Base(models.Model):
  """
  Base parent class for all the models
  """
  timestamp = models.DateTimeField(blank=True, db_index=True)
  is_active = models.BooleanField(default=True, db_index=True)

  def __init__(self, *args, **kwargs):
    super(Base, self).__init__(*args, **kwargs)

  class Meta:
    abstract = True

class OttPlatform(Base):
  """
  """

  name = models.CharField(max_length=200)
  ott_type = models.CharField(max_length=50)

  def __str__(self):
    return self.name

So, this helps you to stop duplication of code, but there is one more issue we can handle here. The is_active column is used to mark the row as deleted. Mainly in our use case, we can't delete the data from the table to keep the track of changes. So is_active field helps us with that. But now we have to use the is_active filter in every query.

We can solve this by overriding the manager, let see how


# First, define the Manager subclass.
class AtiveOTTManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(is_active=True)

class OttPlatform(Base):
  """
  """

  name = models.CharField(max_length=200)
  ott_type = models.CharField(max_length=50)
  
   # the order matters, first come default manager, then custom managers.
  objects = models.Manager() # The default manager.
  active_objects = AtiveOTTManager() # The active OTT manager.

  def __str__(self):
    return self.name

# Now you have to do OttPlatform.active_objects.all(), to get all the active OTT platform name.

So, with overriding the manager we don't have to write a filter for is_active in every query.

Cheers!

#100DaysToOffload #django #python