Posts

Showing posts from February, 2025

AWS

Image
AWS Cloud: Aws is a cloud service, cloud means physical servers which are present in your premise or in any geo-location in world from where your application or service can run for your business requirement. Collection of 1000's of servers are known as data center and AWS has multiple data centers across the globe. Below is sample data center image. AWS uses virtualization to share physical resources between users of different geo-location.  These virtual machines come with billing model of usage as we have for electricity and for water supply and usage. Billing modes of fulling the demands are: On-Demand - No upfront payment or long term commitment for server usage by using virtualization. Pay-as-You-Go - Pay on hourly or even second basis for server usage. Cloud computing: It is know as computation service delivered with the help of internet to any end-user. Here in case you are using AWS storage and services and computation in place of your personal or local systems. AWS offers ...

Apache Spark and PySpark

Image
 Apache Spark and PySpark Some reading materials: Spark Documentation: Spark Official Documentation PySpark Documentation: PySpark API Documentation. Books: "Learning Spark", "Advanced Analytics with Spark"  Big Data: Data volume in TBs, PBs and more. Hadoop is old system which is used for processing bigdata. Hadoop is using file system to process data. Characterisitcs: Volume - size of data, (bytes<kb<mb<gb<tb<pb....) Ex: Digital payments, Social media data, e-commerce data etc... Velocity - Data speed of travel Variety - Excel, RDBMS, txt, JSON, XML, HTML, Documents, images, audio, video, geo maps, etc... Veracity - Quality of data and accuracy of data, trust worthy data or not. Value - Actionable information or any data which will provide meaning for business decision or can be considered as useful. Company requirements: Data storage Data processing speed Scalability Hadoop: (HDFS - filesystem) (Map Reduce(computation) - Programming framework) Hand...

VS Code

Image
                            VS Code a generic IDE You can purchase the course from Udemy for VS Code. All these notes are from same course. Boost your productivity with Visual Studio Code, the best code editor of the moment! Completely updated in 2024!   Best part of this course is that it is for lifetime and will never expire and will get update as well whenever developer of course think it is needed. You can download the software form VS code official website. After installing VSCODE in Windows you can check the vscode version and command to launch it through cmd.  Cmd command to check VS code version: Cmd command to launch VS code from present folder: VS Code image: cmd command for launching VS Code with specific location:   VS Code image and folder details: VS Code Details: It provide details about workspace status, action bar, Folder structure, workspace and terminals part.  Exp...

Data Structure and Algorithm

Image
                                         Linked List Introduction to Linked List: We have arrays concept in programming then why we need to know about LinkedList. As array is continous memory allocation and variables can be created on different memory location hence there is a chance that we will loose memory space for continous array creation. To overcome this issue we need to have a datatype which can act as Array but won't be using contionous memory for storing data. Graphical representation of issue and solution: Array representation in memory:      Linked List representation as an object for memory: