Module 2 BigDataframework(Final)

EXECUTIVE M. TECH IN BLOCKCHAIN AND BIG DATA FIRST SEMESTER BIGDATA FRAMEWORKS 1

Prefix 2

CONTENT UNIT - 2: JAVA IN THE CLOUD...........................................................................................4 3

UNIT - 2: JAVA IN THE CLOUD STRUCTURE 2.0 Learning Objectives 2.1 Introduction 2.2 Overview of Java as a programming language 2.2.1 History and Evolution of Java 2.3 Java And Cloud Computing 2.3.1 Key Features and Benefits of Java 2.4 Suitability Of Java for Cloud 2.5 Set Up A Java Development Environment In The Cloud 2.6 Summary 2.7 Keywords 2.8 Learning Activity 2.9 Unit End Questions 2.10 References 4

2.0 LEARNING OBJECTIVES After studying this unit, you will be able to: • Analyse the role of Java in Cloud Computing • Realize the benefits of Java Programming • Hands on in installing the Cloud environment 2.1 INTRODUCTION Java is a popular programming language that is widely used for cloud computing. Its popularity is due to its scalability, portability, and ability to run on any platform that has a Java Virtual Machine (JVM) installed. Java is an object-oriented language that is used for developing applications that can run on a wide range of platforms, including cloud computing. Java is well-suited for cloud computing because it is designed to be platform- independent. This means that Java applications can run on any platform that has a JVM installed, which includes cloud computing platforms. Java's ability to run on multiple platforms makes it a great choice for cloud computing, as it can be used to develop applications that can run on different cloud computing platforms, such as Amazon Web Services, Microsoft Azure, or Google Cloud. Additionally, Java has a vast collection of libraries and frameworks that can be used to develop cloud-based applications. These libraries and frameworks make it easier to develop cloud-based applications by providing pre-built code that can be reused in applications. Some of the popular Java libraries and frameworks for cloud computing include Spring Framework, Hibernate, Apache Struts, and Apache Tomcat. Java also provides a secure environment for cloud computing, which is essential for protecting data and ensuring privacy in the cloud. Java's security features include automatic memory management, class loaders, and byte-code verification. These features help prevent security breaches, such as buffer overflow attacks and malware infections. 2.2 OVERVIEW OF JAVA AS A PROGRAMMING LANGUAGE Java is a general-purpose, high-level programming language that is widely used for developing a range of applications, including desktop, web, and mobile applications. It was 5

first introduced by James Gosling, Patrick Naughton, and Mike Sheridan at Sun Microsystems in 1995. Since then, it has become one of the most popular programming languages in the world, used by millions of developers to build complex and sophisticated applications. Java is an object-oriented language, which means that it is based on the concept of objects. An object is a software entity that represents a real-world object and has attributes and behaviors. This enables it to handle any large-scale to very large-scale systems at ease. As one of the most popular programming languages in the world, Java has been utilized by millions of developers for a variety of applications. Over the years, Java has undergone numerous updates and advancements, making it one of the most versatile and powerful programming languages available today. One of the areas in which Java has seen significant growth and adoption is in the field of cloud computing. With its powerful features and ability to work seamlessly with cloud technologies, Java has proven to be an ideal language for developers looking to build cloud-based applications. 2.2.1 HISTORY AND EVOLUTION OF JAVA Java was first introduced in 1995 by James Gosling and his team at Sun Microsystems. It was designed as an object-oriented programming language with a syntax similar to C++. Java was designed to address some of the limitations of C++, such as its lack of memory management, platform dependence, and lack of built-in support for network programming. One of the key features that set Java apart from other programming languages of its time was its ability to run on any platform with a Java Virtual Machine (JVM). This made Java an ideal language for developing cross-platform applications that could run on any operating system or hardware platform. Over the years, Java has undergone numerous updates and advancements. The language has evolved to include new features and functionalities, such as support for lambda expressions, functional programming, and modular programming. The latest version of Java, Java 20, was released in March 2023, and it includes new features such as pattern matching, records, and sealed classes. 6

2.3 JAVA AND CLOUD COMPUTING Cloud computing refers to the delivery of computing services, including servers, storage, databases, software, analytics, and more, over the Internet. These services can be accessed from anywhere in the world, and users often pay only for the resources they use. Cloud computing has revolutionized the way that applications are developed, deployed, and managed. Cloud computing allows developers to build applications that can be deployed and scaled quickly and easily, without the need for significant infrastructure investments. Java is an ideal language for developing cloud-based applications, as it has several key features that make it well-suited for this purpose. 2.3.1 KEY FEATURES AND BENEFITS OF JAVA: Java has several key features and benefits that make it an ideal language for developing cloud-based applications. Some of the key features and benefits of Java include: • Object-Oriented Programming: Java is an object-oriented programming language, which means that it is designed around the concept of objects. This makes it easy to write reusable code and build complex applications. • Platform Independence: Java code can run on any platform with a Java Virtual Machine (JVM). This makes it easy to write cross-platform applications that can run on any operating system or hardware platform. • Garbage Collection: Java has built-in support for garbage collection, which means that developers do not need to manually manage memory allocation and deallocation. This makes it easier to write bug-free code and improves the performance of the application. • Scalability: Java's support for multithreading and concurrency makes it easy to build applications that can scale to meet the demands • High Performance: Java programs are highly optimized, which makes them fast and efficient. Java achieves high performance through the use of just-in-time (JIT) compilation, which compiles Java code into native machine code at runtime. • Multithreaded: Java has built-in support for multithreading, which enables it to take advantage of the processing power of multiple cores. This makes it possible to write concurrent applications that can handle multiple tasks simultaneously. 7

• Dynamic: Java is a dynamic language, which means that it supports dynamic loading of classes and objects at runtime. This makes it easy to write code that can adapt to changing requirements and environments. • Large Standard Library: Java has a large standard library that provides a wide range of tools and resources for developers. This includes libraries for networking, file I/O, database access, and much more. • Open Source: Java is open source, which means that the source code is freely available to developers. This makes it easy for developers to modify and extend the language to suit their needs. Java also has the following advanced features that make it a popular choice for building complex and scalable applications. Here are some of the advanced features of Java: • Generics: Generics are a powerful feature that allows developers to write code that is more reusable and type-safe. With generics, developers can write classes and methods that work with a variety of different types, without having to write separate code for each type. • Lambdas: Lambdas are a feature introduced in Java 8 that allows developers to write more concise and expressive code. Lambdas enable developers to pass code as data, which makes it easier to write functional-style code and perform operations on collections. • Streams: Streams are a feature introduced in Java 8 that allows developers to process collections of data in a functional-style manner. Streams enable developers to perform operations like filtering, mapping, and reducing collections with ease and are a powerful tool for writing efficient and concise code. • Annotations: Annotations are a feature that allows developers to add metadata to their code. Annotations can be used to provide additional information to the compiler, to generate code, or to specify runtime behavior. • Reflection: Reflection is a feature that allows developers to inspect and manipulate objects at runtime. Reflection enables developers to create objects dynamically, inspect the properties and methods of objects, and perform other advanced operations on objects. 8

• Concurrency: Concurrency is a feature that allows developers to write code that can execute multiple tasks simultaneously. Java provides several concurrency mechanisms, such as threads, locks, and semaphores, that make it easier to write scalable and efficient code. • Serialization: Serialization is a feature that allows objects to be converted into a stream of bytes, which can then be transmitted over a network or stored in a file. Java provides a built-in serialization mechanism that makes it easy to serialize and deserialize objects. • Networking: Java is often considered to be the language of the internet. Java provides powerful networking capabilities, including support for HTTP, FTP, and other protocols. With Java's networking features, developers can build applications that communicate with remote servers and other devices. Java's advanced features make it a versatile and powerful language that can be used to build complex and scalable applications. By leveraging features like generics, lambdas, and streams, developers can write code that is more expressive, efficient, and maintainable. Additionally, Java's support for concurrency, serialization, and networking makes it an ideal choice for building distributed systems and networked applications. 2.4 SUITABILITY OF JAVA FOR CLOUD COMPUTING Cloud computing is a model of delivering computing resources over the internet. It involves the use of remote servers, storage, and networking infrastructure to provide on- demand access to a range of computing services, including applications, platforms, and infrastructure. Java is well-suited for cloud computing for several reasons: • Portability: As mentioned earlier, Java is platform-independent, which makes it highly portable. This means that a Java program can run on any platform that has a JVM installed, regardless of the underlying hardware or operating system. This makes it easy to deploy Java applications to the cloud, as they can be run on any cloud platform that supports Java. • Scalability: Cloud computing is all about scalability. It allows organizations to scale their computing resources up or down as needed to meet changing demands. Java is designed with scalability in mind. It has built-in support for multithreading, which enables it to take advantage of the processing power of multiple cores. This makes it 9

possible to scale Java applications horizontally, by adding more instances to a cluster, or vertically, by increasing the resources allocated to a single instance. • Security: Security is a major concern for cloud computing. Java has built-in security features, such as a security manager and a security architecture, which make it easy to write secure applications. Java has a range of security features, such as sandboxing, that help protect against malicious code. Java programs can also be digitally signed, which provides an additional layer of security. • Reliability: Reliability is another key consideration for cloud computing. Java is known for its robustness and stability. It has a garbage collector that automatically frees up memory that is no longer needed by a program. This helps prevent memory leaks and makes Java programs more reliable. Java has built-in support for exception handling, which makes it easy to write robust code that can handle errors and exceptions gracefully. Java programs are also highly optimized, which makes them fast and efficient • Compatibility: Cloud computing involves the use of a range of different technologies and platforms. Java is compatible with a wide range of other languages, technologies, and platforms, which makes it easy to integrate with other cloud services. • APIs: Java has a wide range of APIs (Application Programming Interfaces) that can be used to interact with cloud platforms and services. This includes APIs for cloud storage, databases, messaging, and more. • Microservices: Java is well-suited for developing microservices, which are small, independent components that can be deployed and scaled separately. Microservices architectures are well-suited for cloud computing, as they can be easily distributed across multiple servers or instances. • Containers: Containers are a lightweight, portable way to package and deploy applications. Java supports containers through technologies like Docker and Kubernetes, which make it easier to deploy and manage Java applications in the cloud. • Community: Java has a large and active community of developers, which means that there are plenty of resources available for developers to build complex applications quickly. Java has a rich set of libraries and frameworks that make it easy for developers to build complex applications quickly. Some of the most popular Java frameworks include Spring, Hibernate, and Struts. Some of the most popular Java frameworks include Spring, Hibernate, Struts, etc. 10

2.5 SET UP A JAVA DEVELOPMENT ENVIRONMENT IN THE CLOUD Set up a Java development environment in the cloud To set up a Java development environment in the cloud, you can follow these general steps: • Choose a cloud provider: There are many cloud providers to choose from, such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, and Oracle Cloud. Each provider has its own advantages and pricing models, so choose the one that best fits your needs and budget. When choosing a cloud provider, it's important to consider the following criteria: • Cost: Consider the pricing model of the provider and whether it fits within your budget. Some providers offer pay-as-you-go pricing, while others offer reserved instances for longer-term usage. • Reliability and uptime: Look for providers that offer strong service level agreements (SLAs) and have a track record of high uptime and reliability. Downtime can have a significant impact on your business, so it's important to choose a provider that can offer reliable and consistent service. • Security: Consider the security features of the provider, such as firewalls, encryption, and identity and access management. Look for providers that have strong security measures in place to protect your data and applications. • Scalability: Choose a provider that can scale with your business needs. Look for providers that offer flexible scaling options, such as autoscaling, to ensure that your applications can handle increased traffic and demand. • Performance: Consider the performance of the provider's infrastructure, such as network speeds and latency. Look for providers that offer fast and reliable performance to ensure that your applications can run smoothly. • Support: Look for providers that offer strong customer support, including technical support, documentation, and community forums. Choose a provider that can offer timely and helpful support when you need it. • Integration: Consider how well the provider's services integrate with your existing tools and infrastructure. Look for providers that offer strong integration options and APIs to ensure that you can easily integrate your applications and services. 11

By considering these criteria, you can choose a cloud provider that meets your business needs and requirements. Create a virtual machine (VM): Once you have selected your cloud provider, create a VM instance. This instance will be your virtual computer in the cloud, where you can install and run Java applications. In this tutorial, we are focused on AWS. To create a virtual machine (VM) in AWS, you can follow these general steps: Sign in to your AWS account: - Log in to your AWS account using your credentials. - Open the EC2 console: From the AWS Management Console, navigate to the EC2 console. - Launch a new instance: Click on the \"Launch Instance\" button to launch a new instance. Amazon Machine Image (AMI): Choose an AMI, which is a pre-configured image of an operating system and application server. You can select an AMI provided by AWS or choose one from the AWS Marketplace. Instance Type: Select an instance type, which determines the hardware specifications of your VM, such as CPU, memory, and storage. Configure Instance Details: Configure the instance details, such as the number of instances to launch, the VPC network to use, and the security group settings. Add Storage: Add storage to your instance, such as an Elastic Block Store (EBS) volume, which provides persistent block storage for your instance. Configure Security Groups: Configure security groups to control inbound and outbound traffic to your instance. You can choose to create a new security group or use an existing one. Review And Launch: Review your instance configuration and launch your instance. Connect To Your Instance: Once your instance is launched, you can connect to it using a remote desktop application or a web-based terminal. By following these steps, you can create a VM in AWS and start using it for your applications and services. Connect To The VM: To access your VM and the IDE installed on it, you need to connect to it using a remote desktop application or a web-based terminal. Your cloud provider will provide instructions on how to connect to your VM. 12

To connect to an EC2 instance running Windows, you can use Remote Desktop Protocol (RDP). Follow these steps to connect: Obtain the public IP address of your EC2 instance: Log in to your AWS Management Console and go to the EC2 dashboard. Select your EC2 instance and look for the Public IP address in the Description tab. Configure the security group: By default, EC2 instances running Windows are configured with a security group that allows RDP traffic on port 3389. However, you should verify that the security group associated with your instance allows RDP traffic. Go to the Security Groups section of the EC2 dashboard, select the security group associated with your instance, and verify that it allows inbound traffic on port 3389. Connect to the instance using RDP: On your local machine, search for Remote Desktop Connection or RDP and open it. Enter the public IP address of your EC2 instance and click on Connect. Enter the login credentials: You will be prompted to enter the username and password to log in to the EC2 instance. Use the Administrator username and the password that you specified when launching the instance. Log in to the EC2 instance: After entering the login credentials, click on OK to log in to the EC2 instance. By following these steps, you can connect to your EC2 instance running Windows using RDP and start using it for your Windows-based applications and services. By default, Amazon Linux instances do not come with a graphical user interface (GUI) installed. However, you can install a desktop environment on your EC2 Linux instance and connect to it using a remote desktop client. Here are the steps to connect to an EC2 Linux instance running a desktop environment: Install a desktop environment: You can install any desktop environment of your choice. For example, you can install the Xfce desktop environment by running the following command: sudo yum install -y xfce4 Install a remote desktop client: On your local machine, install a remote desktop client such as VNC Viewer. 13

Install a VNC server on the EC2 instance: Install a VNC server on the EC2 instance. For example, you can install TigerVNC by running the following command: sudo yum install -y tigervnc-server Configure the VNC server: Configure the VNC server to start at boot and set a password for VNC access. Run the following command to start the VNC server and set the password: vncserver :1 This command starts the VNC server on display: 1. Connect to the EC2 instance using VNC Viewer: Open VNC Viewer on your local machine and enter the public IP address of your EC2 instance followed by the VNC display number, separated by a colon. For example, if the public IP address of your EC2 instance is 1.2.3.4 and the VNC display number is 1, enter 1.2.3.4:1 in VNC Viewer and click Connect. Enter the VNC password: Enter the VNC password that you set in Step 4 and click OK. After you connect to the EC2 instance using VNC Viewer, you should be able to see the Linux desktop environment running on the instance. Note: that running a desktop environment on an EC2 instance may consume significant resources and may not be suitable for all use cases. Install Java and Set up environment variables: After creating the VM, you need to install the Java Development Kit (JDK) on it. You can install Java using the command line or by downloading the installer from the Java website or open-source distribution. Once Java is installed, you need to set up environment variables to point to the Java installation directory. This will allow you to run Java commands from anywhere on the VM. To install Java in an EC2 instance running Linux/macOS with OpenJDK, you can follow these steps: Connect to your EC2 instance: Use SSH to connect to your EC2 instance. Update the package manager: Run the following command to update the package manager: sudo yum update Install Java: Run the following command to install Java: For Java 8: sudo yum install java-1.8.0-openjdk-devel 14

For Java 11: sudo yum install java-11-openjdk-devel Verify the Java installation: Run the following command to verify that Java is installed: java -version This should display the version of Java that is installed on your EC2 instance. By following these steps, you can install Java on your EC2 instance and start using it for your Java applications and services. To install Java in an EC2 instance running Windows, you can follow these steps: Connect to your EC2 instance: Use Remote Desktop Protocol (RDP) to connect to your EC2 instance running Windows. Download Java: Download the Java Development Kit (JDK) from the official Oracle website. Install Java: Double-click on the downloaded file to start the installation wizard. Follow the prompts in the wizard to install Java on your EC2 instance. Set the JAVA_HOME environment variable: Once Java is installed, set the JAVA_HOME environment variable to point to the installation directory. To do this, follow these steps: Open the System Properties dialog box: Right-click on the \"Computer\" icon on the desktop and select \"Properties\" from the context menu. Click on the \"Advanced system settings\" link. Click on the \"Environment Variables\" button. Under \"System variables,\" click on the \"New\" button. Enter \"JAVA_HOME\" as the variable name and the path to the Java installation directory as the variable value (e.g., C:\\Program Files\\Java\\jdk1.8.0_291). Click \"OK\" to save the changes. Edit \"PATH\" as the variable name and the path to the JRE installation directory as the variable value (e.g., C:\\Program Files\\Java\\jre1.8.0_291\\bin). Click \"OK\" to save the changes. Verify the Java installation: Open a command prompt and run the following command to verify that Java is installed: 15

java -version This should display the version of Java that is installed on your EC2 instance. By following these steps, you can install Java on your EC2 instance running Windows and start using it for your Java applications and services. Install an Integrated Development Environment (IDE): An IDE is a software application that provides a comprehensive environment for coding, debugging, and testing Java applications. There are several popular IDEs available for Java development, such as Eclipse, IntelliJ IDEA, and NetBeans. You can install an IDE on your VM using the command line or by downloading the installer from the IDE website. In this tutorial, we restrain ourselves to IntelliJ IDE. To install IntelliJ IDEA on an EC2 instance, you can follow these steps: Connect to your EC2 instance: Use SSH to connect to your EC2 instance. Download IntelliJ IDEA: Download the IntelliJ IDEA Community or Ultimate edition from the official JetBrains website. You can download the Linux version of IntelliJ IDEA, as EC2 instances run Linux. Extract the IntelliJ IDEA archive: Navigate to the directory where you downloaded the IntelliJ IDEA archive and run the following command to extract it: tar -xzf ideaIC-<version>.tar.gz Replace <version> with the version number of the IntelliJ IDEA archive that you downloaded. Make sure you have Install Java: Before running IntelliJ IDEA, you need to ensure that Java is installed on your EC2 instance. Follow the instructions above to install Java. Run IntelliJ IDEA: Navigate to the bin directory of the extracted IntelliJ IDEA directory and run the idea.sh script to start IntelliJ IDEA: cd idea-<version>/bin ./idea.sh Replace <version> with the version number of the IntelliJ IDEA archive that you downloaded. Activate IntelliJ IDEA: When you run IntelliJ IDEA for the first time, you will be prompted to activate it. Follow the prompts to activate IntelliJ IDEA or select the option to activate it later. By following these steps, you can install and run IntelliJ IDEA on your EC2 instance and start developing Java applications and services. 16

To install IntelliJ IDEA on an EC2 instance running Windows, you can follow these steps: Connect to your EC2 instance: Use Remote Desktop Protocol (RDP) to connect to your EC2 instance running Windows. Download IntelliJ IDEA: Download the IntelliJ IDEA Community or Ultimate edition from the official JetBrains website. You can download the Windows version of IntelliJ IDEA, as EC2 instances running Windows are similar to regular Windows machines. Install IntelliJ IDEA: Double-click on the downloaded file to start the installation wizard. Follow the prompts in the wizard to install IntelliJ IDEA on your EC2 instance. Install Java: Before running IntelliJ IDEA, you need to ensure that Java is installed on your EC2 instance. Follow the instructions above to install Java. Run IntelliJ IDEA: Once IntelliJ IDEA is installed, double-click on the shortcut icon on your desktop to run it. Activate IntelliJ IDEA: When you run IntelliJ IDEA for the first time, you will be prompted to activate it. Follow the prompts to activate IntelliJ IDEA or select the option to activate it later. By following these steps, you can install and run IntelliJ IDEA on your EC2 instance running Windows and start developing Java applications and services. Start coding: Once you have connected to your VM and launched your IDE, you can start coding Java applications. You can save your code on the VM, and use the IDE to compile and run your applications. Here are the steps to create and run a simple Java program using IntelliJ IDEA: Open IntelliJ IDEA and create a new project: Select File -> New -> Project. In the New Project dialog box, select Java under the left-hand menu and select the JDK version you want to use. Click Next. Choose a project template: You can select a template for your project, such as Java Application or Java Library. For this example, we'll choose Java Application. Click Next. Configure the project: Enter a name for your project and select the project location on your file system. Click Finish. Create a new Java class: Right-click on the src folder in the Project panel and select New -> Java Class. Enter a name for your class and click OK. 17

Write your Java code: In the editor window, you should see a basic Java class with a main method. Write your Java code inside the main method. For example, you can write a \"Hello, World!\" program like this: public class MyFirstJavaProgram { public static void main(String[] args) { System.out.println(\"Hello, World!\"); } } Run the program: Right-click anywhere inside the main method and select Run 'MyFirstJavaProgram.main()'. You should see the output \"Hello, World!\" in the Run panel at the bottom of the IntelliJ IDEA window. Congratulations! You have successfully created and run your first Java program using IntelliJ IDEA. 2.6 SUMMARY Java is a suitable language for cloud computing due to its platform independence, scalability, and portability. Its large library and framework ecosystem, as well as its security features, make it a popular choice for developing cloud-based applications. 2.7 KEYWORD Virtual Machine - A virtual machine (VM) is a computer that has had most of its functionalities virtualized, including the ability to run programs and operating systems. A piece of software known as a hypervisor provides access to computational resources for virtual machines that run on actual hardware. VNC Viewer - For controlling local computers and mobile devices, utilize VNC Viewer. With VNC Viewer software installed, a device like a computer, tablet, or smartphone can connect to and operate a computer in another place. 18

2.8 LEARNING ACTIVITY 1. Define JVM 2.9 UNIT END QUESTIONS A. Descriptive Questions Short Questions 1. Narrate the key benefits of Java for the suitability of Cloud Computing Long Questions 1. Simulate and narrate the steps involved in setting up the cloud environment B. Multiple-Choice Questions 1. Which of the following is not a key feature of Java that makes it suitable for cloud computing? 1. Platform independence 2. Security 3. Portability 4. High performance Answer: d. High performance is not a key feature of Java that makes it suitable for cloud computing. While Java is a performant language, it is not known for being the fastest language for certain use cases. However, Java's platform independence, security, and portability make it well-suited for cloud computing. 2. Which of the following is a popular Java-based framework for building cloud-native applications? a. Spring Boot b. Django c. Ruby on Rails d. Express.js Answer: a. Spring Boot is a popular Java-based framework for building cloud-native applications. 3. Which Java virtual machine (JVM) is used by Amazon Web Services (AWS) to run Java applications in the cloud? a. OpenJDK b. Oracle JDK c. IBM JDK d. Azul Zulu Answer: a. OpenJDK 19

4. Which environmental variable is used to specify the location of the Java executable file? a. JAVA_HOME b. PATH c. CLASSPATH d. JAVA_PATH Answer: b. PATH 5. Which environmental variable is used to specify the installation directory of the Java Development Kit (JDK)? a. JAVA_HOME b. JDK_HOME c. JRE_HOME d. JAVA_PATH Answer: a. JAVA_HOME 2.10 REFERENCES TEXTBOOKS: 1. Mike Frampton, “Mastering Apache Spark”, Packt Publishing, 2015. 2. Tom White, “Hadoop: The Definitive Guide”, O’Reilly, 4th Edition, 2015. 3. Nick Pentreath, Machine Learning with Spark, Packt Publishing, 2015. 4. Mohammed Guller, Big Data Analytics with Spark, Apress, 2015 5. Donald Miner, Adam Shook, “Map Reduce Design Pattern”, O’Reilly, 2012 20