dgplug member blogs

Reader

Read the latest posts from dgplug member blogs.

from nileshpatra

I recently started reading the pym book suggested by folks at #dgplug. Since I have been programming in Python since an year and a half, I could go through the basics fairly quick. Here are the topics I covered:

  • Variable and Datatypes
  • Operators
  • Conditionals
  • Loops
  • Python Datastructures
  • Strings
  • Functions

However, file handling is something I have rarely used till now. This blog talks about the it and some of the great takeaways.

Opening a file

A file can be opened in three modes: ### Read: Opens the file in read-only mode. The file cannot be edited or added content to. The syntax for the same is :

>>> f = open('requirements.txt' , 'r')

### Write: Opens the file in write, you can make desired changes to the file. The syntax for the same is:

>>> f = open('requirements.txt' , 'r')

### Append: Opens file in append mode. You can append further content, but cannot change or modify past content. The syntax for the same is:

>>> f = open('requirements.txt' , 'a')

Reading a file

When a file is openened in read mode, the file pointer is at the beginning of the file. There are different functions for reading the file:

read()

It reads the entire file at once. The file pointer traverses the entire file on calling this function. Therefore, calling this function again will have no effect, since the file pointer is already at EOF. Syntax for the same is:

>>> f.read()
'selenium >= 3.141.0\npython-telegram-bot >= 11.1.0\ndatetime >= 4.3\nargparse >= 1.4.0\nwebdriver-manager >= 1.7\nplaysound >= 1.2.2'

readline()

This function moves the file pointer to the beginning of the next line hence outputting one line at a time. Syntax for readline() function is :

>>> f.readline()
'selenium >= 3.141.0\n'
>>> f.readline()
'python-telegram-bot >= 11.1.0\n'

readlines()

Reads all the lines in a file and returens a list.

>>> f.readlines()
['selenium >= 3.141.0\n', 'python-telegram-bot >= 11.1.0\n', 'datetime >= 4.3\n', 'argparse >= 1.4.0\n', 'webdriver-manager >= 1.7\n', 'playsound >= 1.2.2']

Now, we should always close a file we opened when not in use. Not closing it increases memory usage and degrades the quality of code. Python offers nice functionality to take care of file closing by itself:

with keyword

`with keyword can be used as follows:

>>> with open('requirements.txt' , 'r') as f:
...     f.read()
... 
'selenium >= 3.141.0\npython-telegram-bot >= 11.1.0\ndatetime >= 4.3\nargparse >= 1.4.0\nwebdriver-manager >= 1.7\nplaysound >= 1.2.2'

Writing into a file

The .write() function can be easily used to write into a file. This will place the file pointer to the beginning and over-write the file completely. Here's how that works:

>>> f = open('requirements.txt' , 'w')
>>> f.write('tgbot\n')
6

The return value '6' denotes the number of characters written into the file

Hope you enjoyed reading the blog, :)

 
Read more...

from nileshpatra

Contributing to open source is one of the best ways to hone up programming skills. Along with writing quality code, using a version control tool plays a crucial role while contributing. There are a lot of source control management platforms such as github , gitlab , phabricator etc. This blog discusses about making code contributions via github.

So what Is a Pull Request?

Pull request, as the name suggests is a patch of code that is sent to original code base to be merged into the source code after review. Usually, maintainers of the project will review the PR(pull request) and merge it into original code base if everything looks okay.

How to make a PR?

1. Fork the Repository and clone

First off, there should be a fork of the upstream repository. Fork is nothing but a copy of the upstream repository onto your own github. This is where you will be pushing your changes. (Since you own it :D) Then, clone of repository so as to do the changes locally and testing them before sending a patch. This should be fairly simple using git clone <repository URL>

2. Make a new branch

Now, if we want to make a change to the source code, we should always ensure that the master/development branch to be always in sync with upstream. You would definitely not like messing the master branch , and if in case the issue's priority is not high, the PR will be pending with the changes in master branch.

Other than that, the master branch is 'supposed' to have the updated code(or the production code), the rest of the features are supposed to be done on separate branches before being pushed into production.

Thus, it it is always a good practice to make new branches for each pull request to be opened. To do this use git checkout -b <branch_name> -t upstream/master This will make the branch in sync with the upstream. If upstream is not added, you can manually add it to remote using git remote add upstream <upstream_URL>

Or alternatively, you can just create a branch and fetch from upstream using:

   git checkout -b <branch_name>
   git fetch upstream

3. Make changes in the created branch and push

make the required changes and commit them via git add and git commit commands After the changes are done, push to your code via git push origin <branch_name>

4. Make a pull request

Usually, just after pushing to github, you would button when you open your repository(on github) clicking on which a PR will be made. It should look something as follows:

PR image

If that doesn't show automatically, navigate to the branch(on github) and make a PR.

That is it! Now keeps doing the requested changes(if asked) locally and keep pushing code on the created branch till the point it is fit for merging.

 
Read more...

from Stories of raydeeam

After resuming my study I've learned about File handling (I can recall file handling in C).

  • File handling Python gives us an easy way to manipulate files. We can divide files in two parts, one is test file which contain simple text, and another one is binary file which contain binary data which is only readable by computer.

    • File opening The key function for working with files in Python is the open() function. The open() function takes two parameters; filename, and mode. There are four different methods (modes) for opening a file:

      "r" - Read - Default value. Opens a file for reading, error if the file does  not exist
      "w" - Write - Opens a file for writing, creates the file if it does not exist
      "a" - Append - Opens a file for appending, creates the file if it does not exist
      "x" - Create - Creates the specified file, returns an error if the file exists
      
    • Creating a file To create a new empty file:

      >>> f = open("file.txt", "x")
      
    • To Create a new file if it does not exist:

      >>> f = open("file.txt", "w")
      
    • Opening a file To open a file we use open() function. It requires two arguments, first the file path or file name, second which mode it should open. If we don't mention any mode then it will open the file as read only.

      >>> f = open ("file.txt")
      >>> f
      <_io.TextIOWrapper name='file.txt' mode='r' encoding='UTF-8'>
      
    • Closing a file After opening a file one should always close the opened file. We use method close() for this.

      >>> f = open ("file.txt")
      >>> f
      <_io.TextIOWrapper name='file.txt' mode='r' encoding='UTF-8'>
      >>> f.close()
      
    • Reading a file To read the whole file at once use the read() method.

      >>> f = open("sample.txt")
      >>> f.read()
      'I am Rayan\nI live in Bangalore\nI am from West Bengal\n'
      

      If we call read() again it will return empty string as it already read the whole file. readline() can help you to read one line each time from the file.

      >>> f = open("sample.txt")
      >>> f.readline()
      'I am Rayan\n'
      >>> f.readline()
      'I live in Bangalore\n'
      

      To read all the lines in a list we use readlines() method.

      >>> f = open("sample.txt")
      >>> f.readlines()
      ['I am Rayan\n', 'I live in Bangalore\n', 'I am from West Bengal\n']
      

      We can loop through the lines in a file object.

      >>> f = open("sample.txt")
      >>> for x in f:
      ...     print(x, end=' ')
      ...
      I am Rayan
      I live in Bangalore
      I am from West Bengal
      

      Example:

      >>> f = open("sample.txt", "w")
      >>> f.write("I am Rayan\nI live in Bangalore\nI am from West Bengal")
      >>> f.close()
      >>> f = open("sample.txt", "r")
      >>> print(f.read())
      I am Rayan
      I live in Bangalore
      I am from West Bengal
      
    • Using the with statement (which I found so cool) In real life scenarios we should try to use with statement. It will take care of closing the file for us.


 
Read more...

from Stories of raydeeam

Today I read about few things those are listed below:

  • Data Structures in Python Data structures is a way to store and organize data. Obviously, some data structures are good in one set of problems, but terrible in other ones. The right choice of data structure can make your code faster, more efficient with memory and even more readable for other human beings. Python has few in-built data structures.

    • Lists List is a sequence of elements. It can store anything: numbers, strings, other lists, functions and etc. The fact that it can store anything is great but it has a disadvantage. This kind of list will require more memory. Let’s take a look at a basic example of list:

      # create list
      >>> list= ['Noname', 'Rayan', 'xyz', 100, 42, 55]
      >>> list
      ['Noname', 'Rayan', 'xyz', 100, 42, 55]
      # check if element is in list
      >>> 42 in list
      True
      
    • Tuples Another way to store a sequence of elements is to use tuples. Tuple is basically the same thing as list but with one difference. You can’t add or remove elements from tuples after initialization. It’s immutable data structure.

      >>> a = ('Noname', 'Rayan', 'xyz', 100, 42, 55)
      a
      >>> ('Noname', 'Rayan', 'xyz', 100, 42, 55)
      
    • Dictionary nother important data structure is dictionary. The difference between dictionary and list is that you access elements in dictionary by key, not by index.

      >>> dict = {'Rayan': 'Das','Kushal': 'Das','Sayan': 'Chowdhury'}
      >>> dict
      {'Rayan': 'Das', 'Kushal': 'Das', 'Sayan': 'Chowdhury'}
      
    • Sets Set stores only unique elements.

      >>> letters = {'a', 'b', 'c'}
      >>> 'c' in letters
      True
      >>> letters.add('d')
      >>> letters
      {'c', 'b', 'd', 'a'}
      
  • Strings In Python we declare strings in between “” or ‘’ or ‘’’ ‘’’ or “”” “”“ There are different methods available for strings.

    • Strip a String Got to know how to strip a string.
  • Functions A function is a block of code which only runs when it is called. You can pass data, known as parameters, into a function. A function can return data as a result.

    • Defining a function This way we can define a function.

      >>> def func(params):
      ...     statement1
      ...     statement2
      
    • Calling a function

      >>> def func():
      ...     print("hello from func")
      ... 
      >>> func()
      hello from func
      
    • Local and Global variables

    • Keyward only arguments We can also mark the arguments of function as keyword only. That way while calling the function, the user will be forced to use correct keyword for each parameter.

    • Docstrings We use docstrings in Python to explain how to use the code, it will be useful in interactive mode and to create auto-documentation.

    • Got to know about Higher-order function. It does at least one of the following step inside: -Takes one or more functions as argument. -Returns another function as output.

    • Map function map is a very useful higher order function in Python. It takes one function and an iterator as input and then applies the function on each value of the iterator and returns a list of results.


 
Read more...

from Stories of raydeeam

I've started Python again from Pym book by Kushal

Read about following topic and solved few basic problems.

  • Data types
  • f-string ( Which I found so cool)
  • Type conversion
  • Conditional statements
  • Loops
    • When I started Python back in college I found it so confusing but then I got to know how it works.

I've paused here. Cleared all my basics again till now. Basics are the key ingredients in long run I believe.


About me

 
Read more...

from Stories of raydeeam

As I said learning is never easy. At least for me. When I first started learning Python from youtube back in my college, Every time tutorials starts with Variables, Operators and Expressions. I thought it's more like C/C++ and then I used to pause the tutorial and procrastinate. I procrastinate so much that I procrastinate the actual procrastination.

Well later I realized that I've wasted the time. But then I've started again and this time through docs. Yes from PYM book and blogs. I think docs gave me a better picture. I'm still learning. It's not that thing that you can complete within a week. Better things take time. Here are few things for beginners who is getting started with Python.

Whitespaces and indentation:

Whitespace and indentation in Python is important. Whitespace in the beginning of the line called indentation. For wrong indentation Python throws an error. Example:

>>> a=10
>>>    b=20
  File "<stdin>", line 1
    b=20
    ^
IndentationError: unexpected indent

There are more places where we should be following the same type of whitespace rules:

  • Add a space after “,” in dicts, lists, tuples, and argument lists and after “:” in dicts.
  • Spaces around assignments and comparisons (except in argument list)
  • No spaces just inside parentheses.

Comments:

Comments are simple English to explain what this code does. It's easier to understand your code if you follow proper commenting for every code snippets. Comment line starts with # and everything after is considered as a comment. Example:

#This is a comment
a = 10
b=20
#This line will add two numbers
a+b

Multi Line Comments:

#This is a comment
#written in
#more than just one line
print("Hey there")

or we can add a multiline string (triple quotes) in our code, and place our comment inside it.

'''
This is a comment
written in
more than just one line
'''
print("Hey there") 

Modules:

Consider a module to be the same as a code library. A file containing a set of functions you want to include in your application. To create a module just save the code you want in a file with the file extension .py To use a module you have to import it first. Example:

>>> import math
>>> math.sqrt(16)
4.0

Keywords and Identifiers:

Following identifiers are used as a keywords and these can not be used as an ordinary identifiers. False class finally is return None continue for lambda try True def from nonlocal while and del global not with as elif if or yield assert else import pass break except in raise

In Python we don’t specify what kind of data we are going to put in a variable. So you can directly write abc = 1 and abc will become an integer datatype. If you write abc = 1.0 abc will become of floating type. Example:

>>> a=10
>>> b=20.0
>>> type(a)
<type 'int'>
>>> type(b)
<type 'float'>

From the above example you can understand that to declare a variable in Python , what you need is just to type the name and the value. Python can also manipulate strings They can be enclosed in single quotes or double quotes like:

>>> 'Python is not a snake'
'Python is not a snake'
>>> "Python is a programming language"
'Python is a programming language'

Input from Keyboard:

Generally the real life Python codes do not need to read input from the keyboard. In Python we use input function to do input.

$ vim hello.py

number = int(input("Enter an integer: "))
if number < 100:
    print("Your number is smaller than 100")
else:
    print("Your number is greater than 100")

Output:

$ ./hello.py
Enter an integer: 229
Your number is greater than 100
$ ./hello.py
Enter an integer: 1
Your number is smaller than 100

Multiple assignments in a single line:

We can assign values to multiple variables in a single line:

>>> i, j = 100, 200
>>> a
100
>>> b
200

Operators and Expressions:

Operators are used to perform operations on variables and values. Python provides few operators which are below listed:

  • Arithmetic operators
  • Assignment operators
  • Comparison operators
  • Logical operators
  • Identity operators
  • Membership operators
  • Bitwise operators

Arithmetic operator:

Arithmetic operators are used with numeric values to perform common mathematical operations:

>>> x=10
>>> y=10
>>> x+y #Addition
20
>>> x-y #Subtraction
0
>>> x*y #Multiplication
100
>>> x/y #Division
1.0
>>> x%y #Modulus
0
>>> x**y #Exponentiation
10000000000
>>> x//y #Floor division
1

Assignment operator:

Assignment operators are used to assign values to variables:

>>> x = 5
>>> x += 5
>>> x -= 5
>>> x *= 5
>>> x /= 5
Comparison operator:

Comparison operators are used to compare two values:

>>> x == y
>>> x != y
>>> x > y
>>> x < y
>>> x >= y
>>> x<= y

Logical operator:

and #Returns True if both statements are true
or #Returns True if one of the statements is true
not #Reverse the result, returns False if the result is true

Identity operator:

is #Returns true if both variables are the same object
is not #Returns true if both variables are not the same object

Membership operator:

in #Returns True if a sequence with the specified value is present in the object
not in #Returns True if a sequence with the specified value is not present in the object

Bitwise operator:

& #AND- Sets each bit to 1 if both bits are 1
| #OR- Sets each bit to 1 if one of two bits is 1
^ #XOR- Sets each bit to 1 if only one of two bits is 1
~ #NOR- Inverts all the bits
<< #Zero fill lest shift (Shift left by pushing zeros in from the right and let the leftmost bits fall off)
>> #Signed fill right shift (Shift right by pushing copies of the leftmost bit in from the left, and let the rightmost bits fall off)

 
Read more...

from Stories of raydeeam

This will guide you through making a pull request to a Git repository through the terminal so that you can make your life easier while working on a project.

How it works:

1. A developer creates the feature in a dedicated branch in their local repo. 2. The developer pushes the branch to a public GitHub repository. 3. The developer files a pull request 4. The rest of the team reviews the code, discusses it, and alters it. 5. The project maintainer merges the feature into the official repository and closes the pull request.

Fork a Repository:

To create a pull request you need to have made your code changes on a separate branch or forked repository. To fork a repository you need to open the repository and click on the fork button. You'll get a copy of the repository after fork. You can work with forked repository made your code changes then create a PR.

Clone the Repository:

To make your own local copy of the repository you would like to contribute to, let's fire up the terminal. We'll use git clone command with the URL that points to your fork of the repository.

$ git clone https://github.com/username/repository.git

Create a Branch:

To avoid trouble later, let's create a new branch in our repo so that the work you'll do is sorted separately.

$ git checkout -b [branch-name]

This command will create a new branch in your repo and switch to it.

Make changes locally:

This is where you'll add your features. If you create a new file remember to add it with git add command and commit them by git commit -m

$ git add [file-name] $ git commit -m [commit-message]

At this point you can use git push command to push the changes to the current branch of your forked repo.

$ git push origin [new-branch]

Make the Pull Request:

This is the most simple step if till now you've done correctly. Now click on the New pull request button in your forked repo. Write down a nice report explaining why these changes should be included in the official source of your project and then confirm. Project author will get a notification that you submitted a PR. They will review your code and you'll get notification for their further actions. They may reject your PR or they may suggest something for changes. Go back, edit it and push again. PR will be automatically updated. If the maintainer is want to integrate your contributions to the project, the maintainer have to click Merge and your code will become a part of the original repo.


 
Read more...

from pradhvan

In the last blog I talked about Iterators and Iterables and I am assuming you're familiar with both of the concepts. So moving forward from let's talk about generators.

Simply put generators are iterators with a yield keyword and they do not return they yield. Similarly, a generator function is one that has a yield keyword in its body.

Let's look at some code and find out a bit more about them so we can define them more formally.

def range_123():
    print("Start")
    yield 1
    yield 2
    yield 3
    print('End')


for number in range_123():
    print(number)
"""
OUTPUT:
Start
1
2
3
End
"""

numbers = range_123() # Assigning generator object to numbers

next(numbers) #Output -> 1
next(numbers) #Output -> 2
next(numbers) #Output -> 3
next(numbers) #Output -> StopIteration Error

When we look closely into the above code range_123() is a generator function. Since generators are iterator we can directly iterate over the whole iterator function or we could assign it to a generator object and then use the next keyword to iterate over it until it's exhausted and raises the StopIteration error in a manner of confirming with the IteratorPrortocal.

Now you must be wondering what is the difference between the yield and return?

  • When a return statement is invoked inside a function, it permanently passes control back to the caller of the function and disposes of a function's local state.

  • When a yield is invoked, it also passes the control back to the caller of the function but it only does so temporarily. It suspends the function and retains its local state.

def greeter(name):
    while True:
        yield f'Hello {name}'

gen_object = greeter('Pradhvan') 
next(gen_object) # Output -> Hello Pradhvan
next(gen_object) # Output -> Hello Pradhvan
next(gen_object) # Output -> Hello Pradhvan
next(gen_object) # Output -> Hello Pradhvan

If we look at the above code we could clearly see that local variable are stashed away temporaily, suspending the function and giving control back to the caller while retaining it's local state.

Since it's doing a lazy evaluation it can be continued anytime with the next() on the generator, which can evaluate somewhat infinitely long series of greeting messages.

Let's look at one more example of a code snippet where multiple yield statements decide the flow of the function.

def repeater():
    while True:
        print("Start")
        yield 1
        yield 2
        print("end")
gen_obj = repeater()
next(iterator) # 1
next(iterator) # 2
next(iterator) # 3

"""
OUTPUT # 1
start 
1
OUTPUT # 2
2
OUTPUT # 3
end
start
1
"""

The above example makes it clear that in a generator function the flow of control of where the function suspends is decided by the yield statement. As the #2 suspends the value at 2 and when we do next() on 3 we get the whole block of statements.

Generator Expression

A generator function can be replaced with a generator expression. These are similar to list comprehensions which that eagerly builds a list, generator expressions return a generator that can lazily produce the items.

def range_123():
    print("Start")
    yield 1
    print("Middle")
    yield 3
    print("End")

res1 = [x*3 for x in range_123()]

"""
Output res1:
Start
Middle
End
"""

for i in res1:
    print("-->",i)
"""output:
--> 3
--> 9
"""
  • The list comprehension eagerly iterates over the items that are to be yielded and prints the Start Middle and End.
  • When the for is iterated over the list produced by the res1, it returns the item that are to be yielded.
def range_123():
    print("Start")
    yield 1
    print("Middle")
    yield 3
    print("End")

res2 = (x*3 for x in range_123())

print(res2) # <generator object <genexpr> at 0x7f8be1d09150>

for i in res2:
    print("-->",i)
"""
Output
Start
-->i
Middle
-->i
End
"""
  • In the case of generator expression, when the for loop iterates over the generator object res2, the body of the generator function range_123() actually executes.
  • Each iteration calls the next() while the iteration advances till a StopIteration is raised.

Since comprehension is a great way to increase the readability of your code and if you're using generator expression, you're making the comprehension more memory efficient.

But sometimes we tend to overuse the whole comprehension feature which backfires, I found a great article Overusing list comprehensions and generator expressions in Python which you should definitely look into.

 
Read more...

from mrinalraj

Learning Unix commands are very essential as it helps you to navigate easily in your directories and files. Today we will learn some basics Unix commands which is easy for any beginners to begin with. Also you will get to know some bonus tips in the end.

Let's begin with creating new file.

cat > filename : creates a new file, if files already exists then overwrites it.

cat >> filename: appends the content in the already existing filename, else if not found then creates a new one.

Now to see if file is created or not.

ls -li : long list the files present in that directory with the inode no.

Now if you want to list files which is in some other directory or same working directory then there comes the concept of Relative addressing and Absolute addressing.

Relative addressing and absolute addressing


Relative addressing: here we can move to the directory which is under that directory.

username@hostname directoryname$ cd directory_name

Absolute Addressing: here we specify address from the root username@hostname directoryname$ cd /home/username/directory_name

Lets play with calendar commands.

cal: prints the calendar of current month

cal mm yyyy : will print calendar of that year of that month.

cal year : will print the calendar of that year

In Unix, you can do calculations as well.

bc: It will open binary calculator. Here you can perform calculations.


ln filename linkname : it creates a new link pointing to the existing file “file_name”

where cp filename newfilename : creates a new file with “newfilename” and then copies the content of the filename into it.

Note ls -li will confirm that in the copy command there will be different inode number while link command creates a new link to file as they have the same inode no.

mkdir is used to make a directory and rmdir is to remove a directory

rm file_name: is used to remove a file_name

Till now we learned some basic tips. Time for some Bonus Tips.

passwd : this command is used to change the current password

sudo passwd -l username : to lock that username from the system

sudo passwd -u username: to unlock that username from the system

sudo passwd -e username: to expire the username password next time he logins he has to give new passwd

sudo passwd -d username: deletes the password of that account and sets that account passwordless.

 
Read more...

from abhinits2046

I was thinking to note down my thoughts so that I can share my experience. So here it is, about how I joined 'dgplug' and get to know about Open-Source.

During my B.Tech 3rd semester somehow I got to know about the word 'Open-Source'. After a year when I was a fifth semester student, I met 'Ratnadeep(rtnpro)' in a tech-talk at my college,talk was over Open-Source , Its opportunities and freedom.Open Source culture was a myth in my college, very few people knew about Open-Source and its freedom of learning. In my batch, hardly fifty percent people know about Open-Source and Communities. During the talk 'Ratnadeep debnath(rtnpro)' mentioned about 'dgplug' summer training program and also told about 'Kushal das(kushal)' and other 'dgplug' operators, in those days I was learning python so after the talk a name was coming again and again in my mind that was 'Kushal das(kushal)', there were many questions and doubts were in my mind e.g. how to start, where to start then I decided to ping 'Ratnadeep debnath(rtnpro)' and he helped me a lot. I was a hindi background student till my 12th standard so I was a little bit shy to talk ans also I was a window user in those days then 'Ratnadeep debnath(rtnpro)' suggested me to shift over fedora, and also suggest to join IRC to get enrolled in 'dgplug' where masterminds like Kushal, Ratnadeep(rtnpro), Sayan, Chandan , Jasonbraganza were there. Summer training program had been started 15 days before i joined. 'Kushal das' suggested me to go through the logs, I did that. Kushal das(kushal) provided his book for python named 'Python you and me', book helped me a lot to understand python with a decent practice. https://pymbook.readthedocs.io/en/latest/ I started digging more about Open-Source and i got my interests in it. At the beginning, after trying on my own, I lost hope because I was struggling to learn to contribute and thought that it's impossible ,I started staying on 'dgplug' after sessions and listened to people what they are taking and used to ask questions frequently whenever doubt appeared in my mind as time passed thing became easier, it's just start for me many thing to go. Jasonbraganza's sessions over reading and writing importance was amazing and encouraged me and many more to write, I am very much thankful to him for his kind help. That's how it all started for me.

 
Read more...

from pradhvan

Iteration is the fundamental technique which is supported by every programming language in the form of loops. The most common one at least from is the For loop and if specifically talk about Python's case, we have For each loop. For each loop are powered by iterators. An iterator is an object that does the actual iterating and fetches data one at a time and on-demand.

Let's take a step back and look back at some of the common terms which would help us in understanding iterators even better.

iterables: anything that can be iterated over is called an iterable.

for item in some_iterable:
    print(item)

sequences: Sequences are iterables which can be indexed.

numbers = [1,2,3,4]
tuples = (1,2,3)
word = 'Hello world'

The iter function

Iter is built-in function and whenever the interpreter needs to iterator over an object, it automatically calls the iter().

The iter() function returns an iterator.

When the iter function is called it does three things:

  1. Checks whether the object implements __iter__ method. (To see this just do dir() on the object.)
  2. If the __iter__ method is not present but the __getitem__is implemented, python creates an iterator that fetches the items in order, starting from the index zero.
  3. If that fails a TypeError is raised stating “ Object is not iterable”.
numbers = [1,2,3,4]
num = iter(numbers) # Builds an iterator 'num' 

Looking at the code snippet above we can make a better definition of an iterable.

*Any object which the __iter__ built-in function can be called an iterable.*

Before moving forward let's look at nifty little way the iter() works with functions to make them work as an iterator.

Let's build a die roller that rolls a die from 1-6 and stops when the die hits 1.

In this usage we need to make sure of two things:

  1. That the iter function must receive a callable that will be invoked every time the next function is called and the callable function should not have any arguments.
  2. The second argument which is called the sentinel which acts as a flag will cause the iterator to raise an exception instead of returning the second argument.
def die_roll():
    return randint(1,6)

roller = iter(die_roll, 1)

print(type(roller)) # <class 'callable_iterator'>

for roll in roller:
    print(roll)

"""
Output:
5
6
3
2
"""

Iterable vs Iterator

Python obtains an iterator from an iterable. Let's look at the for-each loop again to see how everything fits in the picture.

numbers = [1,2,3,4]
for number in numbers:
    print(number)

Looking at the code above we can only see the iterable i.e numbers. But what about the iterator? What about the iter() ? Isn't it suppose to use both to work.

Here we can't see the iterator or the iter() in action but it's working behind the scene. Let's re-write the whole statement in a while loop so we can see how it all fits together.

numbers = [1,2,3,4]
num = iter(numbers) # builds an iterator
while True:
    try:
        print(next(num))
    except StopIteration:
        del num
        break

The flow of the above code is simple:

  1. Iterator num is created from the iterable.
  2. To obtain the value from the iterator next is used.
  3. Iterator raises the StopIteratioin error when there are no further items left.
  4. We delete the iterator and break out of the loop.

You must be wondering everything is fine but why did we delete the iterator.

Iterators have this property that they are one-directional and once all the item is iterated over they can't be reset to the original state.

Thus the StopIteration signals that the iterator is exhausted. Thus it's best to delete it.

Writing your own iterator

Python iterator objects are required to support two methods __iter__ and the __next__ method.

iter method returns self. This allows iterators to be used where an iterable is expected i.e “for” and “in” keywords.

next method returns the next available item, raising the StopIteration when there are no more items to be looped through.

Let's bundle this knowledge and build our very own Range built-in function.

class _Range:
    def __init__(self, start, end, step = 1):
        self.start = start
        self.end = end - 1 
        self.step = step

    def __iter__(self):
        return self

    def __next__(self):
        if self.start > self.end:
            raise StopIteration
        else:
            self.start += self.step
            return self.start - 1

numbers = _Range(1, 3)
print(next(numbers)) # Result -> 1
print(next(numbers)) # Result -> 2
print(next(numbers)) # Raise a StopIteration Exception

Now that we know how an iterator works let's look back at the definition of an iterator again:

*Any object that implements the __next__ no-argument method that returns the next item in a series or raises StopIteration when there are no more items is called an Iterator.*

Just a quick tip before moving forward, the optimal way of creating your own iterator is to make a generator function, not by creating a iterator class like we did here.

Iterator Protocol

The iterator objects are required to support the following two methods, which together form the iterator protocol. The __iter__ and the __next__ method.

iterator.__iter__()
iterator.__next__()
  • Iterator Protocol powers the all the iteration in python.
  • Iterator Protocol also powers the tuple unpacking in Python.
# Tuple unpacking
x,y,z = coordinates
  • Iterator Protocol also powers the star expressions.
numbers = [1,2,3,4,5]
a,b,*rest = numbers 
print(numbers)
  • Most of the built-in functions that require some kind of looping(iterations) in python uses the Iterator Protocol.

Python's tongue twister

Iteratorables are not necessarily iterators but an iterator is necessarily iterable.

Example: Generators are iterators that can be looped over but lists are iterables but not an iterator.

Reasons to use Iterator:

  • Iterators allow lazy evaluation possible which saves memory.
  • Iterators allow for infinitely long iterables.

Not so common iterators

  • Enumerate objects are also iterators.
  • Zip objects are also iterators.
  • Reversed objects are iterators.
  • Files are also iterators.
letters = ['a','b','c','d']
next(enumerate(letters)) # Result -> (0, 'a')
next(zip(letters,letters)) #  Result -> ('a','a')
next(reversed(letters)) #  Result -> 'd'
next(open('iterator.txt')) #  Result -> 'iterator\n'
 
Read more...

from nileshpatra

I have been using git from quite some time, but there are certain amazing things about git that I learnt just recently.

1. Blaming with git blame

git blame gives the commit hash, author and time for every line in a file. This can be used to know who are the people who have contributed to that particular code and the commit hash too! It can also help figure out who added a particular line in a project which broke it, and then go blame them :P

2. Rectifying via git commit —amend

Ever been in a hurry and made a spelling mistake? That's exactly what is command is for! This can change the commit message for the last commit. Just type git commit --amend that would open the file in the default terminal text editor. Change the message, save and quit, and done!

3. Squashing

Squash is not a command, but definitely a concept in itself. Imagine having changes in a file multiple times, which would make a lot of unnecessary commits. Now if any point of time, one needs to go back in history, doing that would be just plain painful with several commits cluttered. Squashing here comes to rescue! Squashing means convert related commits to one commit. It's nothing but a kind of interactive rebase. for last n commits you can do git rebase -i HEAD~n. Here's how it goes: squash1

This will open the commits in the default text editor. Instead of pick in front of commit, type squash for those commits you wish to squash.

file

Save and exit, you will get another file which asks the user to choose final commit messages. Type the appropriate message and delete other messages. Save an quit, it's squashed! \o/

4. Stashing

While working on projects, things are in a messy state and you want to switch branches for a bit to work on something else. Now, you do not want to commit things half done(you need to commit everything before switching). git stash comes to rescue! Stashing takes the current state of your working directory and pushes it onto a stack of unfinished changes. Now you can switch branches! You can also apply the unfinished changes any time with git stash apply

5. Tagging

Tagging is a feature in git. For every software there are releases, now imagine if a software is released and development team is working for the next release. Meanwhile, there is a bug report with severe priority. Now, hat needs to be fixed in the previous version! Remembering the last commit for each release is not a good idea. Git here offers the functionality! It has the ability to tag specific points in a repository’s history as being important(releases). There are five basic commands:

  • git tag – will list all the tags in the project.
  • git checkout <tag_version> – jumps to the repository's state with the particular tag version.
  • git tag -a <version> -m '<commit message> – creates a tag with the version number and commit message.
  • git tag -d <tag_name> – deletes the tag from the list
  • 'git push origin —tags` – pushes tags to the origin

6. Applying patches

A lot of open source work is done via mailing list. The kernel mailing list comes at the top of my head. A lot of commits are mailed as patches. These patches contain the diffs(git diff). They can be easily applied via git apply <filename>. Now the code can be tested with the patch applied.

7. Short git status

Git has a short status flag so you can see your changes in a more compact way. If you run git status -s or git status –short you get a more simplified output from the command. status-short The ?? indicates the file hasn't been staged. D indicates deleted file and there are a few other tags too.

8. Cloning can be done under a different name

You don't need to have a weird name that the upstream would have to your repository. You can always rename the directory while cloning. The syntax is: git clone <upstream URL> <directory name>

Git is indeed a wonderful tool in itself. There's so much to learn. Hope you enjoyed reading the blog, :)

 
Read more...

from nileshpatra

Ever wondered to get a GUI application from another machine get rendered onto yours? If not, Linux offers that to you :P Let's see how: First of all we shall discus some basic terminologies before moving ahead:

The display server

The display server is very similar to a web-server. So Imagine that you have an apache server. Now when a client requests a service, the server would send a few instructions accordingly to the client over http/https protocol. The client(which here is the web browser) will render those instructions and display the content requested. The display server in Linux works almost the same way. The server here interacts with the hardware (CPU) and sends instructions over a protocol known as the 'X' protocol. The latest version of the same is 11, and hence it's called X11.

X-Forwarding

Think what would happen if the display-server is on one machine and client on the other machine? Yes! that would render the graphical applications onto the client machine. That's is exactly what X-forwarding is – render the application on one machine while it runs on a different machine.

Okay, enough of terminologies, let's get to implementing it now.

On the server, configure /etc/ssh/sshd_config to contain the following:

X11Forwarding yes
X11DisplayOffset 10

These two are usually commented out, ensure that the sshd_config contains these. You may need to restart ssh for the changes to be picked up. You can run this if the ssh server doesn't pick that up and then restart the server:

cat /var/run/sshd.pid | xargs kill -1

Next, start ssh on both the machines. On Debian or Debian derivatives you can do so by doing:

sudo service ssh start

Now we will connect the client to the server. On the client machine type:

ssh -X <server name>@<server ip>

If you are facing trouble finding out ip, you can do so via:

nmcli -p device show

And it's done! You have SSH'd into the server machine. Now you can render GUI apps on the client machine, while it runs on the server.

firefox

Here's me rendering firefox on the client :D

 
Read more...

from pradhvan

Context Managers in Python help the users to manage the resources efficiently in a program i.e opening or closing a resource or locking or unlocking a resource. Context Managers come handy when we have to manage resource before and after an event.

The most common context manager that every python programmer uses very frequently for managing files, is a with as statement.

with open('MyContextManager.txt') as f:
    f.write('Using Context Manager is FUN !')

The above code snippet has mainly two advantages:

  • Helps to close resources automatically and effectively. This is a small code block, so it could be observed easily that the file was opened and closed properly but what would when the scope of the function increases? This is where context managers really come into the picture.
  • Makes the code readable and complex logic simple. The above code can also be written as:
file = open('MyContextManager.txt','W')
try:
    file.write('Not using a Context Manager.')
finally:
    file.close()    

Here we manage the opening and closing of a file manually with a try-finally statement.

Python's standard library comes with a module, contextlib. This contains utilities for working with context managers and the with statement.

Writing Your Own Context Manager

So why would someone want to write their own Context Managers?

Because, Context Managers are best at managing resources before and after an event; thus one doesn't have to worry about the allocation/de-allocation or Locking/Unlocking or Opening/Closing of events.

Also, they make the code simple and more readable.

Writing your own context manager can be done in two ways; either create your own class or use the Contextlib module to create a Context Manager decorator.

Let's first look at how we can create a simple Context Manager class. A Context Manager class consists of two main methods enter and exit. If you're familiar with testing, you can compare these two methods with the setup and teardown.

Just like every class in Python, the init method is optional. But in the case of Context Managers, we use init only if we're using a with as statement. init has to be passed the name which you want to associate with as in the with as statement.

Now let's take a look at a simple Game of Thrones inspired ContextManager which creates a dict of the house symbols.

class ThronesContextManager:
    def __init__(self, HouseSymbol):
        self.HouseSymbol = HouseSymbol

    def __enter__(self):
        print("Enter: {}".format(self.HouseSymbol)")
        return self.HouseSymbol

    def __exit__(self, *exc):
        print("Exit: {}".format(self.HouseSymbol))


with ThronesContextManager({"House Stark": "Wolf"}) as HouseSymbol:
    HouseSymbol["Targaryen"] = "Three Headed Dragon"
    
"""
---Output---
Enter: {'House Stark': 'Wolf'}
Exit: {'House Stark': 'Wolf', 'Targaryen': 'Three Headed Dragon'}
"""
  • The init method takes in the dict associated with the as, similar to as what is done in the with-as statement. It creates an instance of the class and assigns it to the dict. Much similar to any normal Python Class.
  • The enter method is called by the with and is passed the dict. It returns the value which is associated with the dict(HouseSymbol).
  • The exit takes in the exception(*exc), these are of mainly three types exc: exception, exctype: exception type and exctb: exception_traceback.
  • If for some reason you want the program to ignore the exception you can also return True to just ignore the exception.

Now taking a look at the above code example we can say that any Context Manager has two methods an enter method and an exit method.

Before moving forward to contextmanager decorator let's break down the code snippet we saw in the starting of the post and see how it works behind the hood.

Since we know how context managers work it won't be difficult to the observe what's happening when we call with as statement while opening a file.

with open('MyContextManager.txt') as f:
    f.write('Using Context Manager is FUN !')
  1. With calls the enter method of the File class.
  2. The enter method opens the file and returns it.
  3. The opened file handle is passed to f.
  4. We write to the file using .write().
  5. The with statement calls the exit method.
  6. The exit checks for exceptions, if no exception is found it closes the file.

The easier way to write a context manager is by using the Contextlib module and creating a context manager decorator.

The good thing about using the @contextmanager is that it builds the enter and exit method for you automatically, thus we can transform a generator function into a contextmanager decorator.

Let's re-write the ThronesContextManager again but with a @ThronesContextManager.

from contextlib import contextmanager

@contextmanager
def ThronesContextManager(data):
    print("Enter: {}".format(data))
    yield data 
    print("Exit: {}".format(data))

with ThronesContextManager({"House Stark": "Wolf"}) as HouseSymbol:
    HouseSymbol["House Targaryen"] = "Three Headed Dragon"
    
"""
---Output---
Enter: {'House Stark': 'Wolf'}
Exit: {'House Stark': 'Wolf', 'House Targaryen': 'Three Headed Dragon'}
"""

PyRandom

Here are some interesting things I found about Contextmanagers. I came across these while researching for this blog post and hence the that's the reason I am adding this to the section PyRandom. I would keep updating this section as I learn more about Context Managers.

  • Context Managers do not create a separate new scope in the program i.e variables defined inside the withas block will be available after the block is executed.
with open('MyContextManager.txt') as f:
    # Variable defined inside the Context Manager
    VariableName = f.read()
print(VariableName)
  • When using multiple ContextManager in a withas statement the flow of enter and exit statement becomes LIFO(Last In First Out) i.e the enter method that is called last will have it's exit method called first.
import contextlib

@contextlib.contextmanager
def make_context(name):
    print ('entering:', name)
    yield name
    print ('exiting :', name)

with make_context('A') as A, make_context('B') as B, make_context('C') as C:
    print ('inside with statement:', A, B, C)
    
"""
---OUTPUT---
entering: A
entering: B
entering: C
inside with statement: A B C
exiting : C
exiting : B
exiting : A
"""

What now ?

Since we covered all the basic stuff on Context Managers, we can start digging deeper and learn to use Context Managers in a more realistic scenarios. So here are a few things that I would like you to read more about:

  • Learn how to handle exceptions in/with Context Managers.
  • Try to find out real project use cases where using a Context Manager would be best suited.
  • Find out the role of init and enter in the Context Manager.

Still can't get enough ?

The reason behind the blog is that I recently picked a Python problem which goes something like this

Write a Context Manager named Suppress which suppresses the exception of a given type/types i.e if the given exception type is raised, that exception should be caught and muted in a sense.

Code Example:

>>> x = 0
>>> with suppress(ValueError):
...     x = int('hello')
...
>>> x
0
>>> with suppress(ValueError, TypeError):
...     x = int(None)
...
>>> x
0

Since you read this far I am assuming you are also just starting to learn about this topic. Let's put it all that we have learned so far to a test and get some hands-on experience of writing your Context Manager. Try to solve this problem.

I am still solving the problem and once it's done I would link my solution here.

Happy Coding !

 
Read more...