initial
This commit is contained in:
nm1
commit
74ae93a96e
5 changed files with 94691 additions and 0 deletions
|
|
@ -0,0 +1,99 @@
|
|||
Basics
|
||||
https://www.youtube.com/watch?v=kTp5xUtcalw&t=335s : freecodecamp ones
|
||||
https://www.youtube.com/playlist?list=PLinedj3B30sDvBfeK9EPz9pcJNlM0f3ph : piyush garg ones
|
||||
https://www.youtube.com/watch?v=pR5jOQnfNtY : docker in mobile
|
||||
|
||||
### Microservices architecture
|
||||
- A variant of the service oriented architecture(SOA) structural style - arranges an aplication as a collection of loosely coupled services
|
||||
- In a microservices architecture, services are fine-grained and the protocols are lightweight.
|
||||
|
||||
|
||||
### Monolithic architecture
|
||||
- built as a single unt
|
||||
- deployed as a single unit (building time would be high)
|
||||
- duplicated on each server
|
||||
- ex: 3-tier apps like MVC architecture
|
||||
|
||||
|
||||
### Microservices
|
||||
- Breaking big system into smaller parts and each with its own responsibility.
|
||||
- Scale these smaller pieces independtly from each other
|
||||
- Loosely coupled : No strong bond because we expose functionality though api
|
||||
- Enable autonomous development by different teams, languages : Can be written by smaller teams so outsourcing and independent development is possible so they can even use different programming languages too.
|
||||
- Each microservices can have its own separate database
|
||||
|
||||
|
||||
### Diagrams
|
||||
Monolithic architecture
|
||||
|
||||
|
||||
For deploying in more servers
|
||||
|
||||
|
||||
|
||||
Microservice architecture
|
||||
|
||||
|
||||
For scaling
|
||||
|
||||
|
||||
Monolith to microservices
|
||||
|
||||
|
||||
|
||||
## Microservice anti patterns
|
||||
|
||||
### Challenges
|
||||
- Risk of unneccessary complexities when converting monlithic to so many microservices
|
||||
- numerous services can be impacted too
|
||||
- Security of microsercices is also a challenge
|
||||
|
||||
|
||||
### Microservices Benefits
|
||||
- Improved fault isolation : wont take whole system down if one of them go down
|
||||
- less vendor lock in or technology in bcz of use of open source technologies
|
||||
- Easier to understand
|
||||
- Smaller and faster development
|
||||
- Scalibility
|
||||
|
||||
### Microservices - Drawbacks
|
||||
- Complexity is added to resolve complexity issues
|
||||
So need to start it slowly with a pace that everyone can keep it
|
||||
- Its easier to test independent services but it is also necessary to test whole system to check if they r communicating with each other properly or not.
|
||||
- Is deployment really simpler
|
||||
- challenges with multiple times
|
||||
- One microservice update other microservices too
|
||||
- Need to have multiple databases too
|
||||
- Latency issues : As there r so many routes needs to be taken as each microservice will call api to talk to each other so there will be latency issues too
|
||||
- Transient errors : Sometimes between calls there maybe errors which can be fixed with retry so make sure to do that
|
||||
- Multiple point of failure
|
||||
- Security issues as when communicating we may not want to expose everything and needed to be take care of that
|
||||
|
||||
|
||||
|
||||
## Cloud Natives
|
||||
- Cloud natives uses containers, service meshes, microservices, immutable infrastructure and declaritive APIs.
|
||||
Immutable infrastructure means instead of updating it we replace with newer versions.
|
||||
These technologies is used to build and run scalable applications in modern, dynamic environments such as public, private and hybrid cloud.
|
||||
- use of metrics, creation and updation are automated and frequent deployment of eye impact changes
|
||||
- using of open source projects
|
||||
|
||||
|
||||
### Cloud native concepts
|
||||
- Speed and agility
|
||||
- Application architecture
|
||||
|
||||
- Mentality : Pets vs Cattle
|
||||
- Unlike VM where we need to take care it, infrastructure in this immutable and disposable like cattle
|
||||
- Provisioned in minites and destroyed on demand
|
||||
- Never updated or repaired, just re-provisioned/replaced.
|
||||
- Cloud Native Trail Map
|
||||
- Breaks journey into measurable objects
|
||||
- Containerization
|
||||
- Done through docker containers
|
||||
- CI/CD
|
||||
- Setup Continuous Integration/Continuous Delivery (CI/CD) so changes to source code automatically result in a new container being built, tested and deployed to staging and eventually perhaps to production
|
||||
- Orchestration : Eg : Kubernetes and deploy using help chart
|
||||
- Obervability and analysis : Monitoring, logging and tracing .
|
||||
- Service proxy, discovery and mesh : Eg :envoy, CoreDNS, LinkerD
|
||||
- Networking, Policy and Security: Eg : CNI, Open policy agent, Falco
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Containers
|
||||
|
||||
A container is a unit of deployment. It contains everything for a code to run. We take the container, push them on server and it should run. Container contains code, runtime, system tools and system libraries. It can have external dependencies too like database or cache
|
||||
|
||||
Reason of using container
|
||||
- Move faster by deploying smaller units
|
||||
- Use few resources
|
||||
- Fit more into the same host
|
||||
- Faster automation when using CI/CD techniques
|
||||
- Portability
|
||||
- Isolation
|
||||
|
||||
Virtual machine vs containers
|
||||
- VM runs on hardware where OS is installed. OS hypervisor lets us create virtual machine where we install an OS . So we would have bios and firmware too as it is hardware even though it is virtual. VM is basically **virtualize the hardware**
|
||||
- In containers we still have hardware and OS. Container runtime is installed in the OS instead of creating virtual machine.Container images are run in the memory. Container dont need to boot as it uses the HOST kernel and they dont have to boot. They will need small disk space as there is no OS with few megabytes of hard disk and 64 or 128MB of RAM required.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### More
|
||||
Containers are made up of layers
|
||||
|
||||
Eg : `docker pull` command will download container image and each layer will be downloaded individually and each layer has unique id. If a layer already exist then it wont download again as docker uses local cache to store container's images.
|
||||
Advantage is if we download version 2 then it wont download whole image, just the layers which is not present
|
||||
So a goal is to create docker container images which smallest number of layers possible
|
||||
Layers in the docker : Top one is both read-write but lower ones in read-only.
|
||||
|
||||
|
||||
### Container registry
|
||||
- Centralized container repository
|
||||
- Think Github but for containers.
|
||||
- Docker hub is a great example
|
||||
- Major providers have container registry services like AWS, azure and GCP.
|
||||
|
||||
|
||||
**Orchestrator**
|
||||
- Allow us to Manage, scale monitor the containers that we run on servers
|
||||
- We can use our own like kubernates, swarm, service fabric
|
||||
- or we can use a managed cluster offered by cloud providers.
|
||||
|
||||
146
joplin notion extension/Web Development/Extras/Docker/Docker.md
Normal file
146
joplin notion extension/Web Development/Extras/Docker/Docker.md
Normal file
|
|
@ -0,0 +1,146 @@
|
|||
Docker
|
||||
- Docker is both the company and platform
|
||||
- An open source container runtime
|
||||
- Cross platform desktop support
|
||||
- Command line tool
|
||||
- "Dockerfile" file format for building container images
|
||||
- Windows let us create both windows and linux containers
|
||||
|
||||
**Docker cli commands**
|
||||
- `docker info` : display system information
|
||||
- `docker version`
|
||||
- `docker login` : by default it will login to dockerhub
|
||||
|
||||
Docker running commands
|
||||
- `docker run [imageName]` : execute image in memory. image is not present in cache, it will be downloaded automatically.
|
||||
there is another flag named `--name` to give name to docker instance which will run. if we dont provide docker will generate random name.
|
||||
eg : `docker run --name web [imageName] ` here web is the name
|
||||
**imposing limits on docker instance**
|
||||
`docker run --memory="256m" nginx`
|
||||
`docker run --cpus=".5" nginx`
|
||||
- `docker run -d [imageName]` : to run container in background so u can still use the terminal
|
||||
- `docker stop [containerName]` : will stop the execution but it will still in memory
|
||||
- `docker kill [containerName]` : to remove container in memory which r stuck
|
||||
- `docker image inspect` : useful for debugging
|
||||
- **Note**: When we use run we create a container from the image. Also when we use flags with `run` certain configuration are set too To stop execution we can use stop command but using start we can again start it with same configuration too.
|
||||
|
||||
|
||||
|
||||
**Usage**
|
||||
|
||||
**Attach shells**
|
||||
|
||||
Working command
|
||||
`docker exec -it web3 bash`
|
||||
here `web3` is container name which is running in background
|
||||
for executing commands for background running containers
|
||||
`podman exec web3 bash -c "ls"`
|
||||
for non running containers
|
||||
`docker run -it --name archc8 archlinux:latest /usr/bin/bash`
|
||||
for non running containers so basically creating a new container
|
||||
`docker run -it alpine:latest sh -c "apk add --update bash && exec bash"`
|
||||
By using the `--rm` flag with either the docker run or docker create command, you ensure that the container is automatically removed once it stops
|
||||
|
||||
**to start distro and keep it running**
|
||||
`podman run -itd --name ubuntu1 ubuntu /bin/bash`
|
||||
**Clearing things up**
|
||||
|
||||
For removing containers, they must be stopped first
|
||||
|
||||
|
||||
|
||||
## Docker build containers
|
||||
- 
|
||||
- `docker build -t ngthml:1.0 dockers/` (dockers folder contain Dockerfile)
|
||||
- If Dockerfile name is not DockerFile then specify that name with `-f` flag.
|
||||
- Eg of dockerfile
|
||||
```text
|
||||
FROM nginx:alpine
|
||||
COPY . /usr/share/nginx/html
|
||||
```
|
||||
FROM command specify base image and when we build something we use image which already exist. nginx:alpine is very lightweight
|
||||
then copy whole folder containing Dockerfile to `/usr/share/nginx/html` of alpine
|
||||
- Nodejs Dockerfile
|
||||
alpine is very lightweight and there isnt even bash installed so we cant have interactive shell too. But we can install bash by
|
||||
`docker exec -it node_hello apk add --update bash`
|
||||
or including it in `RUN apk add --update nodejs npm bash`
|
||||
```text
|
||||
FROM alpine
|
||||
RUN apk add --update nodejs npm
|
||||
COPY . /src
|
||||
WORKDIR /src
|
||||
RUN npm install
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["node", "./app.js"]
|
||||
```
|
||||
- RUN apk .... : Using RUN we can execute commands when generating container. in this case using apk package manager of alpine we are installing nodejs and npm.
|
||||
Then we copy current folder to `/src` and making it as work directory too.
|
||||
- `WORKDIR` is a combination or cd and mkdir as we dont need to make directory too as workdir will handle it. `RUN cd` wont have effect as when using RUN commands will be run in different shell so thats why using WORKDIR is preferred
|
||||
WORKDIR / changes the working directory for future commands.
|
||||
RUN cd / has no impact on the subsequent RUNs.
|
||||
- EXPOSE will make sure which port shoould be accessible by other containers or host system
|
||||
- ENTRYPOINT means commands that will run when the container start
|
||||
**Difference between run,cmd and entrypoint**
|
||||
https://www.bmc.com/blogs/docker-cmd-vs-entrypoint/
|
||||
https://www.cloudbees.com/blog/understanding-dockers-cmd-and-entrypoint-instructions
|
||||
- RUN. Mainly used to build images and install applications and packages, RUN builds a new layer over an existing image by committing the results.
|
||||
- CMD. Sets default parameters that can be overridden from the Docker Command Line Interface (CLI) when a container is running. eg : `cmd node app.js`
|
||||
- ENTRYPOINT. Default parameters that cannot be overridden when Docker Containers run with CLI parameters.
|
||||
**Example**
|
||||
if there is `cmd node app.js` and
|
||||
`docker run nodecontainer node app2.js`
|
||||
then in docker container it will be
|
||||
`node app2.js` as it will override
|
||||
|
||||
if there is `entrypoint ["node", "app.js"]` and
|
||||
`docker run nodecontainer node app2.js`
|
||||
then in docker container it will be
|
||||
```bash
|
||||
node app.js
|
||||
node app2.js
|
||||
```
|
||||
instead of overriding `node app2.js` is appended
|
||||
- docker tag
|
||||
- Assign a name and tag to a specific image ID:
|
||||
`docker tag id name:tag`
|
||||
|
||||
- Assign a tag to a specific image:
|
||||
`docker tag image:current_tag image:new_tag`
|
||||
- docker cp
|
||||
to copy files from docker container to host
|
||||
`docker cp <containerId>:/file/path/within/container /host/path/target`
|
||||
eg:
|
||||
`docker cp node_hello_sh:/ alpine_node/`
|
||||
|
||||
to copy files from host to docker container
|
||||
`docker cp /host/path/target container_id:/file/path/within/container`
|
||||
eg:
|
||||
`docker cp foo.txt container_id:/foo.txt`
|
||||
- docker logs
|
||||
`docker logs <container_id>`
|
||||
or when creating container
|
||||
`docker run --log-level=debug ...`
|
||||
- Expose port of already created container
|
||||
While it is not possible to expose port we can access it by `http://container_ip:PORT`. we can find container ip by
|
||||
`docker inspect container_name | grep IPAddress` . run `docker ps` to find container name first
|
||||
- environment variables when starting a container from image
|
||||
`docker run --name my-tomcat -p 127.0.0.1:8080:8080 -e APP_DB_DB=mydb -e APP_DB_USER=dbuser -e APP_DB_PASS=dbpass --link mongo-myapp:mongo -v /path-to/tomcat/webapps:/usr/local/tomcat/webapps -d tomcat:8-jre8-alpine`
|
||||
- creating image from container
|
||||
after making some changes in container we can create its own image too
|
||||
`docker commit nginx_base` where nginx_base is name of container
|
||||
this will create image but it have no tag so crating its tag with
|
||||
`docker tag 0c17f0798823 hello_world_nginx`
|
||||
or simply add tag when creating image
|
||||
`docker commit nginx_base hello_world_nginx`
|
||||
here hello_world_nginx is name of image
|
||||
- docker volume
|
||||
`docker run -v $PWD:/data -it archivebox/archivebox init --setup`
|
||||
`-v $PWD:/data`: This mounts the current working directory (represented by $PWD, which is a shell variable) on your host machine to the /data directory inside the Docker container. This is crucial because it allows ArchiveBox to store its data (archived websites, indexes, etc.) persistently on your host machine, even if the container is stopped or removed. The -v flag stands for "volume."
|
||||
- docker import and export images
|
||||
https://stackoverflow.com/questions/24482822/how-to-share-my-docker-image-without-using-the-docker-hub
|
||||
Based on this blog, one could share a docker image without a docker registry by executing:
|
||||
`docker save --output latestversion-1.0.0.tar dockerregistry/latestversion:1.0.0`
|
||||
Once this command has been completed, one could copy the image to a server and import it as follows:
|
||||
`docker load --input latestversion-1.0.0.tar`
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
podman
|
||||
https://www.youtube.com/watch?v=Z5uBcczJxUY : Stop Using Docker. Use Open Source Instead
|
||||
|
|
@ -0,0 +1,118 @@
|
|||
projects
|
||||
|
||||
## spring quiz app
|
||||
- location : `/home/hpes/Videos/imp/java/spring/genz/quiz_back_docker-compose`
|
||||
**following will not work**
|
||||
- to go to bash use this
|
||||
`docker run --rm -it --name quiz-app quiz-app-docker:1.0 bash`
|
||||
- run app (use -d for detach mode)
|
||||
`docker run -p 8085:8080 --rm --name quiz-app quiz-app-docker:1.0`
|
||||
- build
|
||||
`docker build -t quiz-app:1.0 .`
|
||||
- Dockerfile
|
||||
```
|
||||
FROM alpine:latest
|
||||
RUN apk add --update bash openjdk17 maven mariadb mariadb-client
|
||||
COPY . /spring-app
|
||||
WORKDIR /spring-app
|
||||
RUN mvn package
|
||||
# Initialize MySQL database (this step will only run the first time)
|
||||
RUN mysql_install_db --user=root --basedir=/usr --datadir=/var/lib/mysql
|
||||
RUN mysqld_safe --user=root & \
|
||||
sleep 3 && \
|
||||
mysql -e "create database quiz;" && \
|
||||
mysql quiz < quiz.sql
|
||||
EXPOSE 8080
|
||||
RUN chmod +x entrypoint.sh
|
||||
# Start the MySQL service and your backend
|
||||
CMD sh /spring-app/entrypoint.sh
|
||||
|
||||
```
|
||||
|
||||
**docker compose method** (1.3 gb)
|
||||
docker-compose.yml
|
||||
```yml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
mysql:
|
||||
image: mysql:latest # You can use a different version if necessary
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: strong_password
|
||||
ports:
|
||||
- "3307:3306" # Expose MySQL on port 3307
|
||||
volumes:
|
||||
- /var/lib/docker/volumes/mysql_app_data/_data:/var/lib/mysql # Persist MySQL data
|
||||
|
||||
spring-app:
|
||||
build: .
|
||||
environment:
|
||||
SPRING_DATASOURCE_URL: jdbc:mysql://mysql:3306/quiz?allowMultiQueries=true
|
||||
SPRING_DATASOURCE_USERNAME: root
|
||||
SPRING_DATASOURCE_PASSWORD: strong_password
|
||||
ports:
|
||||
- "8085:8080" # Expose Spring Boot app on port 8080
|
||||
depends_on:
|
||||
- mysql # Ensure MySQL starts before Spring Boot app
|
||||
|
||||
```
|
||||
|
||||
Dockerfile for spring-app
|
||||
```
|
||||
FROM alpine:latest
|
||||
RUN apk add --update bash openjdk17 maven
|
||||
COPY . /spring-app
|
||||
WORKDIR /spring-app
|
||||
RUN mvn package
|
||||
EXPOSE 8080
|
||||
|
||||
|
||||
CMD java -jar target/demo-0.0.1-SNAPSHOT.jar
|
||||
|
||||
```
|
||||
|
||||
two commands
|
||||
`docker-compose build` and `docker-compose up`
|
||||
|
||||
|
||||
## youtube playlist downloader
|
||||
200 mb
|
||||
- location : `/home/hpes/Videos/imp/nodejs/production/youtube downloader/`
|
||||
- `docker build -t ytdown:1.0 .`
|
||||
DockerFile
|
||||
```
|
||||
FROM alpine
|
||||
RUN apk add --update nodejs npm bash
|
||||
|
||||
# App 1
|
||||
WORKDIR /src/puppetter_test
|
||||
COPY puppetter_test/ ./
|
||||
RUN npm install
|
||||
|
||||
# App 2
|
||||
WORKDIR /src/ytDownloadSpace
|
||||
COPY ytDownloadSpace ./
|
||||
RUN npm install
|
||||
|
||||
|
||||
# App 3
|
||||
WORKDIR /src/ytdlApi
|
||||
COPY ytdlApi/ ./
|
||||
RUN npm install
|
||||
|
||||
|
||||
|
||||
|
||||
RUN npm install -g pm2
|
||||
|
||||
#config file
|
||||
WORKDIR /src/puppetter_test
|
||||
EXPOSE 8081
|
||||
CMD pm2 start ecosystem.config.js --no-daemon
|
||||
|
||||
# CMD ["node", "app.js"]
|
||||
|
||||
```
|
||||
|
||||
for vps
|
||||
[setup.md](../../../../_resources/setup.md)
|
||||
Loading…
Reference in a new issue