En busca de Wally

¿A quién no le interesa encontrar a esta buscada ballena en medio de un océano tan grande? Pues a nosotros no la verdad, preferimos buscar otro tipo de cosas como archivos y directorios en nuestro sistema. Para ello haremos uso de los comandos find(1), locate(1), whois(1),whereis(1).

Diferencias

find(1) a diferencia de locate(1) es un comando que busca a tiempo real y tiene muchísimas funcionalidades añadidas como filtrar por nombre, tipo de ejecutable, fecha...

19 May 2018 8:11pm GMT

NoSQL databases are every bit as popular today as more conventional, relational ones. One of the most popular NoSQL systems is Apache Cassandra. It's designed to deal with big data, and can be scaled across large numbers of servers. This makes it resilient and highly available.

This package is relatively new on Fedora, since it was introduced on Fedora 26. The following article is a short tutorial to set up Cassandra on Fedora for a development environment. Production deployments should use a different set-up to harden the service.

Install and configure Cassandra

The set of database packages in Fedora's stable repositories are the client tools in the cassandra package. The common library is in the cassandra-java-libs package (required by both client and server). The most important part of the database, the daemon, is available in the cassandra-server package. Some more supporting packages may be listed by running the following command in a terminal.

dnf list cassandra\*

First, install and start the service:

$ sudo dnf install cassandra cassandra-server
$ sudo systemctl start cassandra

To enable the service to automatically start at boot time, run:

$ sudo systemctl enable cassandra

Finally, test the server initialization using the client:

$ cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.1 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cqlsh> CREATE KEYSPACE k1 WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> USE k1;
cqlsh:k1> CREATE TABLE users (user_name varchar, password varchar, gender varchar, PRIMARY KEY (user_name));
cqlsh:k1> INSERT INTO users (user_name, password, gender) VALUES ('John', 'test123', 'male');
cqlsh:k1> SELECT * from users;

 user_name | gender | password
-----------+--------+----------
      John |   male |  test123

(1 rows)

To configure the server, edit the file /etc/cassandra/cassandra.yaml. For more information about how to change configuration, see the the upstream documentation.

Controlling access with users and passwords

By default, authentication is disabled. To enable it, follow these steps:

1. By default, the authenticator option is set to AllowAllAuthenticator. Change the authenticator option in the cassandra.yaml file to PasswordAuthenticator:

authenticator: PasswordAuthenticator

2. Restart the service.

$ sudo systemctl restart cassandra

3. Start cqlsh using the default superuser name and password:

$ cqlsh -u cassandra -p cassandra

4. Create a new superuser:

cqlsh> CREATE ROLE <new_super_user> WITH PASSWORD = '<some_secure_password>' 
    AND SUPERUSER = true 
    AND LOGIN = true;

5. Log in as the newly created superuser:

$ cqlsh -u <new_super_user> -p <some_secure_password>

6. The superuser cannot be deleted. To neutralize the account, change the password to something long and incomprehensible, and alter the user's status to NOSUPERUSER:

cqlsh> ALTER ROLE cassandra WITH PASSWORD='SomeNonsenseThatNoOneWillThinkOf'
    AND SUPERUSER=false;

Enabling remote access to the server

Edit the /etc/cassandra/cassandra.yaml file, and change the following parameters:

listen_address: external_ip
rpc_address: external_ip
seed_provider/seeds: "<external_ip>"

Then restart the service:

$ sudo systemctl restart cassandra

Other common configuration

There are quite a few more common configuration parameters. For instance, to set the cluster name, which must be consistent for all nodes in the cluster:

cluster_name: 'Test Cluster'

The data_file_directories option sets the directory where the service will write data. Below is the default that is used if unset. If possible, set this to a disk used only for storing Cassandra data.

data_file_directories:
    - /var/lib/cassandra/data

To set the type of disk used to store data (SSD or spinning):

disk_optimization_strategy: ssd|spinning

Running a Cassandra cluster

One of the main features of Cassandra is the ability to run in a multi-node setup. A cluster setup brings the following benefits:

• Fault tolerance: Automatically replicates data to multiple nodes for fault-tolerance. Also, it supports replication across multiple data centers. You can replace failed nodes with no downtime.
• Decentralization: There are no single points of failure, no network bottlenecks, and every node in the cluster is identical.
• Scalability & elasticity: Can run thousands of nodes with petabytes of data. Read and write throughput both increase linearly as new machines are added, with no downtime or interruption to applications.

The following sections describe how to setup a simple two-node cluster.

Clearing existing data

First, if the server is running now or has ever run before, you must delete all the existing data (make a backup first). This is because all nodes must have the same cluster name and it's better to choose a different one from the default Test cluster name.

Run the following commands on each node:

$ sudo systemctl stop cassandra
$ sudo rm -rf /var/lib/cassandra/data/system/*

If you deploy a large cluster, you can do this via automation using Ansible.

Configuring the cluster

To setup the cluster, edit the main configuration file /etc/cassandra/cassandra.yaml. Modify these parameters:

• cluster_name: Name of your cluster.
• num_tokens: Number of virtual nodes within a Cassandra instance. This option partitions the data and spreads the data throughout the cluster. The recommended value is 256.
• seeds: Comma-delimited list of the IP address of each node in the cluster.
• listen_address: The IP address or hostname the service binds to for connecting to other nodes. It defaults to localhost and needs to be changed to the IP address of the node.
• rpc_address: The listen address for client connections (CQL protocol).
• endpoint_snitch: Set to a class that implements the IEndpointSnitch. Cassandra uses snitches to locate nodes and route requests. The default is SimpleSnitch, but for this exercise, change it to GossipingPropertyFileSnitch which is more suitable for production environments:
  • SimpleSnitch: Used for single-datacenter deployments or single-zone in public clouds. Does not recognize datacenter or rack information. It treats strategy order as proximity, which can improve cache locality when disabling read repair.
  • GossipingPropertyFileSnitch: Recommended for production. The rack and datacenter for the local node are defined in the cassandra-rackdc.properties file and propagate to other nodes via gossip.
• auto_bootstrap: This parameter is not present in the configuration file, so add it and set to false. It makes new (non-seed) nodes automatically migrate the right data to themselves.

Configuration files for a two-node cluster follow.

Node 1:

cluster_name: 'My Cluster'
num_tokens: 256
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
        - seeds: 10.0.0.1, 10.0.0.2
listen_address: 10.0.0.1
rpc_address: 10.0.0.1
endpoint_snitch: GossipingPropertyFileSnitch
auto_bootstrap: false

Node 2:

cluster_name: 'My Cluster'
num_tokens: 256
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
        - seeds: 10.0.0.1, 10.0.0.2
listen_address: 10.0.0.2
rpc_address: 10.0.0.2
endpoint_snitch: GossipingPropertyFileSnitch
auto_bootstrap: false

Starting the cluster

The final step is to start each instance of the cluster. Start the seed instances first, then the remaining nodes.

$ sudo systemctl start cassandra

Checking the cluster status

In conclusion, you can check the cluster status with the nodetool utility:

$ sudo nodetool status

Datacenter: datacenter1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens       Owns    Host ID                               Rack
UN  10.0.0.2  147.48 KB  256          ?       f50799ee-8589-4eb8-a0c8-241cd254e424  rack1
UN  10.0.0.1  139.04 KB  256          ?       54b16af1-ad0a-4288-b34e-cacab39caeec  rack1
 
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless

Cassandra in a container

Linux containers are becoming more popular. You can find a Cassandra container image here on DockerHub:

centos/cassandra-3-centos7

It's easy to start a container for this purpose without touching the rest of the system. First, install and run the Docker daemon:

$ sudo dnf install docker
$ sudo systemctl start docker

Next, pull the image:

$ sudo docker pull centos/cassandra-3-centos7

Now prepare a directory for data:

$ sudo mkdir data
$ sudo chown 143:143 data

Finally, start the container with a few arguments. The container uses the prepared directory to store data into and creates a user and database.

$ docker run --name cassandra -d -e CASSANDRA_ADMIN_PASSWORD=secret -p 9042:9042 -v `pwd`/data:/var/opt/rh/sclo-cassandra3/lib/cassandra:Z centos/cassandra-3-centos7

Now you have the service running in a container while storing data into the data directory in the current working directory. If the cqlsh client is not installed on your host system, run the one provided by the image with the following command:

$ docker exec -it cassandra 'bash' -c 'cqlsh '`docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cassandra`' -u admin -p secret'

Conclusion

The Cassandra maintainers in Fedora seek co-maintainers to help keep the package fresh on Fedora. If you'd like to help, simply send them email.

Photo by Glen Jackson on Unsplash.

18 May 2018 12:26pm GMT

Happy #FaaSFriday today we are going to learn how to build our very own serverless microblogging backend using Pure Ruby. Alex Ellis of OpenFaaS believes it would be useful to not rely on any gems that require Native Extensions (MySQL, ActiveRecord, bcrypt, etc…) so thats what I intend to do, This tutorial will use nothing byt pure ruby to keep containers small.

Before you continue reading I highly recommend you checkout OpenFaaS on Twitter and Star the OpenFaas repo on Github

Disclaimer

This microblogging platform as is should not be used in a production environment as password encryption is sub-optimal (i.e its plain SHA2 and not salted) but.. it could be changed easily something more suitable like bcrypt.

We make extensive use of Environment variables however You should use secrets read more about that here.

Now thats out of the way lets get stuck in.

Assumptions

As always in order to follow this tutorial there are pre-requisites.

• You have a working OpenFaaS deployment
  • Mine is on Kubernetes, but you can also use Docker Swarm
• You have an understanding of Ruby
• You have access to a mySQL Database
• You can make calls to an OpenFaaS Function
  • faas-cli
  • cURL

What is OpenFaas?

With OpenFaaS you can package anything as a serverless function - from Node.js to Golang to CSharp, even binaries like ffmpeg or ImageMagick.

Architecture

Our serverless microblogging platform consists of 5 Functions

Register

Our Register function will accept a JSON String containing a Username, Password, First Name, Last Name and E-Mail address. Once submitted we will check the Database for an existing Username and E-Mail if none is found we will add a new record to the database.

Login

Our Login function will accept a JSON string containing a username and password, We will then return a signed JWT containing the username, first name, last name and e-mail.

Add Post

Our Add Post function will accept a JSON string containing the token(from login) and body of the post.

Before we add the post to the database we will call the Validate JWT function to validate the signature of the JWT Token. We will take the User ID from the token.

List Posts

When we call the list posts function we can call it with a JSON string to filter the posts by user and Paginate. By Default we will return the last 100 posts. To paginate we will add an offset parameter to our JSON.

If we don't send a filter we will return the latest 100 posts in our database.

Registering Users

In order to add posts we need to have users, Lets get started by registering our first user.

Run the following SQL Query to create the users table.

CREATE TABLE users
(
 id int PRIMARY NOT NULL AUTO_INCREMENT,
 username TEXT KEY NOT NULL
 password TEXT NOT NULL,
 email TEXT KEY NOT NULL,
 first_name text NOT NULL,
 last_name int NOT NULL
);
CREATE UNIQUE INDEX users_id_uindex ON users (id);

Now we have our table lets create our first function.

$ faas-cli new faas-ruby-register --lang ruby

this will download the templates and create our first function. Open the Gemfile and add the 'ruby-mysql' gem

source 'https://rubygems.org'

gem 'ruby-mysql'

thats the only Gem we need for this function. ruby-mysql is a pure ruby implementation of a mySQL connector and suits the needs of our project. We will be using this gem extensively.

Now open up handler.rb and we add the following code.

require 'mysql'
require 'json'
require 'digest/sha2'

class Handler
    def run(req)
      @my = Mysql.connect(ENV['mysql_host'], ENV['mysql_user'], ENV['mysql_password'], ENV['mysql_database'])
      json = JSON.parse(req)
      if !user_exists(json["username"], json["email"])
        password = Digest::SHA2::new << json['password']
        stmt = @my.prepare('insert into users (username, password, email, first_name, last_name) values (?, ?, ?, ?, ?)')
        stmt.execute json["username"], password.to_s, json["email"], json["first_name"], json["last_name"]
        return "{'username': #{json["username"]}, 'status': 'created'}"
      else
        return "{'error': 'Username or E-Mail already in use'}"
      end
    end

    def user_exists(username, email)
      @my.query("SELECT username FROM users WHERE username = '#{Mysql.escape_string(username)}' OR email = '#{Mysql.escape_string(email)}'").each do |username|
        return true
      end
      return false
    end
end

And thats our function, Lets have a look at some of this function in detail. In our runner I declared a class variable @my with our mySQL connection. I then parsed the JSON we passed to the function. I used a user_exists method to determine if a user exists in our database, if not I moved on to create a new user. I hashed the password with SHA2 and used a prepared statement to insert our new user.

Open your faas-ruby-register.yml and make it match the following, Please ensure you use your own image instead of mine if you are making modifications.

provider:
  name: faas
  gateway: http://127.0.0.1:8080

functions:
  faas-ruby-register:
    lang: ruby
    handler: ./faas-ruby-register
    image: affixxx/faas-ruby-register
    environment:
      mysql_host: <HOST>
      mysql_user: <USERNAME>
      mysql_password: <PASSWORD>      
      mysql_database: <DB>

Now lets deploy and test the function!

$ faas-cli build -f faas-ruby-register.yml # Build our function Container
$ faas-cli push -f faas-ruby-register.yml # Push our container to dockerhub
$ faas-cli deploy -f faas-ruby-register.yml # deploy the function
$ echo '{"username": "affixx", "password": "TestPassword", "email":"caontact@keiran.scot", "first_name": "keiran", "last_name":"smith"}' | faas-cli invoke faas-ruby-register
{'username': username, 'status': 'created'}
$ echo '{"username": "affixx", "password": "TestPassword", "email":"caontact@keiran.scot", "first_name": "keiran", "last_name":"smith"}' | faas-cli invoke faas-ruby-register
{'error': 'Username or E-Mail already in use'}

awesome register works. Lets move on.

Logging In

Now we have a user in our database we need to log them in, This will require generating a JWT token so we need to generate an RSA Keypair. On a unix based system run the following commands.

$ openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:2048
$ openssl rsa -pubout -in private_key.pem -out public_key.pem

Now we have our keypair we need a base64 representation of the text, using a method of your choice get a base64 representation.

Lets generate our new function with faas-cli

$ faas-cli new faas-ruby-login --lang ruby

Before we start working on the function lets get the faas-ruby-login.yml file ready

provider:
  name: faas
  gateway: http://127.0.0.1:8080

functions:
  faas-ruby-login:
    lang: ruby
    handler: ./faas-ruby-login
    image: affixxx/faas-ruby-login
    environment:
      public_key: <BASE64_RSA_PUBLIC_KEY>
      private_key: <BASE64_RSA_PRIVATE_KEY>
      mysql_host: mysql
      mysql_user: root
      mysql_password:
      mysql_database: users

Now we can write the function. This one is a little more complex than registration. So open the faas-ruby-login/handler.rb file and replace it with the following.

require 'mysql'
require 'jwt'
require 'digest/sha2'

class Handler
    def run(req)
      my = Mysql.connect(ENV['mysql_host'], ENV['mysql_user'], ENV['mysql_password'], ENV['mysql_database'])
      token = nil
      req = JSON.parse(req)
      username = Mysql.escape_string(req['username'])
      my.query("SELECT email, password, username, first_name, last_name FROM users WHERE username = '#{username}'").each do |email, password, username, first_name, last_name|
        digest = Digest::SHA2.new << req['password']
        if digest.to_s == password
          user = {
            email: email,
            first_name: first_name,
            last_name: last_name,
            username: username
          }

          token = generate_jwt(user)
          return "{'username': '#{username}', 'token': '#{token}'}"
        else
          return "{'error': 'Invalid username/password'}"
        end
      end
      return "{'error': 'Invalid username/password'}"
    end

    def generate_jwt(user)
      payload = {
        nbf: Time.now.to_i - 10,
        iat: Time.now.to_i - 10,
        exp: Time.now.to_i + ((60) * 60) * 4,
        user: user
      }

      priv_key = OpenSSL::PKey::RSA.new(Base64.decode64(ENV['private_key']))

      JWT.encode(payload, priv_key, 'RS256')
    end
end

The biggest difference between this function and our register function is of course the JWT generator. JWT (JSON Web Tokens) are an open, industry standard RFC 7519 method for representing claims securely between two parties.

Our payload obviously contains our user hash after we fetched this from the database, However there are some other fields required.

nbf: Not Before, Our token is not valid before this timestamp. We subtract 10 seconds from the timestamps to account for clock drift.
iat: Issued at, This is the time we issued the token, Again we set this to 10 seconds in the past to account for time drift.
exp: Expiry, this is when our token will no longer be, we have it set to 14400 seconds (4 hours).

Lets test our login function!

$ echo '{"username": "affixx", "password": "TestPassword"}' | faas invoke faas-ruby-login
{'username': 'affixx', 'token': 'eyJhbGciOiJSUzI1NiJ9.eyJuYmYiOjE1MjY1OTMyMTgsImlhdCI6MTUyNjU5MzIxOCwiZXhwIjoxNTI2NjA3NjI4LCJ1c2VyIjp7ImVtYWlsIjoiY2FvbnRhY3RAa2VpcmFuLnNjb3QiLCJmaXJzdF9uYW1lIjoia2VpcmFuIiwibGFzdF9uYW1lIjoic21pdGgiLCJ1c2VybmFtZSI6ImFmZml4eCJ9fQ.qchkmOk8dsrw7SL6Rhi0nHyIlaHX4pzUNXXAQMEOb6IU0n1uT9AJEhFVptZ7tueriaTauY1zmYjKm79pd_UfekVICU4EMbGKt8bQaWrmlqpSel88PyQwolI_bYZqybW2TwWYsdwHcGgGgfb8A8ssk9y6YhktviKdofQYPUmLmaB5uljFHkMvNIg-ByJQpTYmCnMfAC-JF6mOsh65dKCP3qz78HiSX3gHODG1Gk1OJbePVpyDNmw7pGrO97c7kUgTWs5wVmD7Kgs697tAkPz65pFDavwZHSvdzpPEZ47Bh8NCGfWe73KYpceCjmOZK6tuawIx0MM4YP0XWke7kOtKkg'}

Success! lets check the token on JWT.io

Success we have a valid JWT.

What happens with an invalid username/password

$ echo '{"username": "affixx", "password": "WrongPassword"}' | faas invoke faas-ruby-login
{'error': 'Invalid username/password'}

Exactly as expected!

Thats all folks!

Thanks for reading part 1 of this tutorial, Check back next #FaaSFriday when we will work on posting!

As always the code for this tutorial is available on github, there are also extra tools available!

The post #FaasFriday - Building a Serverless Microblog in Ruby with OpenFaaS - Part 1 appeared first on Keiran.SCOT.

18 May 2018 8:00am GMT

J'avais, par le passé, fait un article sur mon passage d'Eclipse vers Netbeans. Je dois dire qu'au fil du temps, cet IDE m'a déçu : Cycle de vie assez long. Plugin Python qui n'est plus maintenu depuis près de 3 ans. Lourd (qui a dit Java ?). Oracle, qui via sa politique de rachat se […]

Cet article Atom pour remplacer Netbeans est apparu en premier sur Guillaume Kulakowski's blog.

17 May 2018 6:52pm GMT

EPIC Planning Document

History / Background

Since 2007, Fedora Extra Packages for Enterprise Linux (EPEL) has been rebuilding Fedora Project Linux packages for Red Hat Enterprise Linux and its clones. Originally the goal was to compile packages that RHEL did not ship but were useful in the running of Fedora Infrastructure and other sites. Packages would be forked from the nearest Fedora release (Fedora 3 for EPEL-4, Fedora 6 for EPEL-5) with little updating or moving of packages in order to give similar lifetimes as the EL packages. Emphasis was made on back-porting fixes versus upgrading, and also not making large feature changes which would cause confusion. If a package could not longer be supported, it would be removed from the repository to eliminate security concerns. At the time RHEL lifetimes were thought to be only 5-6 years so back-porting did not look like a large problem.

As RHEL and its clones became more popular, Red Hat began to extend the lifetime of the Enterprise Linux releases from 6 years to 10 years of "active" support. This made trying to back-port fixes harder and many packages in EPEL would be "aged" out and removed. This in turn caused problems for consumers who had tied kick-starts and other scripts to having access to those packages. Attempts to fix this by pushing for release upgrade policies have run into resistance from packagers who find focusing on the main Fedora releases a full time job already and only build EPEL packages as one-offs. Other attempts to update policies have run into needing major updates and changes to build tools and scripting but no time to do so. Finally, because EPEL has not majorly changed in 10 years, conversations about changing fall into "well EPEL has always done it like this" from consumers, packagers, and engineering at different places.

In order to get around many of these resistance points with changing EPEL, I suggest that we frame the problems around a new project called Extra Packages for Inter Communities. The goal of this project would be to build packages from Fedora Project Linux releases to various Enterprise Linux whether they are Red Hat Enterprise Linux, CentOS, Scientific Linux or Oracle Enterprise Linux.

Problems and Proposals

Composer Limitations:

Problem:

Currently EPEL uses the Fedora build system to compose a release of packages every couple of days. Because each day creates a new compose, the only channels are the various architectures and a testing where future packages can be tested. Updates are not in a separate because EPEL does not track releases.
EPEL packagers currently have to support a package for the 10 year lifetime of an RHEL release. If they have to update a release, all older versions are no longer available. If they no longer want to support a package it is completely removed. While this sounds like it increases security of consumers, Fedora does not remove old packages from older releases.

Proposed Solution

EPIC will match the Enterprise Linux major/minor numbers for releases. This means that a set of packages will be built for say EL5 sub-release 11 (aka 5.11). Those packages would populate for each supported architecture a release, updates and updates-testing directory. This will allow for a set of packages to be composed when the sub-release occurs and then stay until the release is ended.

/pub/epic/releases/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/
/pub/epic/updates/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/
/pub/epic/updates/testing/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/
/pub/epic/development/5/CR/

Once a minor release is done, the old tree will be hard linked to an appropriate archive directory.

/pub/archives/epic/releases/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/
/pub/archives/epic/updates/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/
/pub/archives/epic/updates/testing/5/5.11/{x86_64,source,i386,aarch64,arm,ppc64}/

A new one will be built and placed in appropriate sub directories. Hard links to the latest will point to the new one, and after some time the old-tree will be removed from the active directory tree.

/<package_name>/epic/6/10/
/<package_name>/epic/6/11/
/<package_name>/epic/7/6/
/<package_name>/epic/7/7/</package_name>

/<package_name>/epic-6/6.10/
/<package_name>/epic-6/6.11/
/<package_name>/epic-7/7.6/
/<package_name>/epic-7/7.7/</package_name>

/<package_name>/epic-6.10/
/<package_name>/epic-6.11/
/<package_name>/epic-7.6/
/<package_name>/epic-7.7/

Git module patterns will need to match what upstream delivers for any future EL.

Continuous Integration (CI) Gating

EPIC-6

The EL-6 life-cycle is reaching its final sub releases with more focus and growth in EL-7 and the future. Because of this gating will be turned off EPIC-6. Testing of packages can be done at the packagers discretion but is not required.

EPIC-7

The EL-7 life-cycle is midstream with 1-2 more minor releases with major API changes. Due to this, it makes sense to research if gating can be put in place for the next minor release. If the time and energy to retrofit tools to the older EL are possible then it can be turned on.

EPIC-next

Because gating is built into current Fedora releases, there should be no problem with turning it on for a future release. Packages which do not pass testing will be blocked just as they will be in Fedora 29+ releases.

Modules

EPIC-6

Because EL-6's tooling is locked at this point, it does not make sense to investigate modules.

EPIC-7

Currently EL-7 does not support Fedora modules and would require updates to yum, rpm and other tools in order to do so. If these show up in some form in a future minor release, then trees for modules can be created and builds done.

EPIC-next

The tooling for modules can match how Fedora approaches it. This means that rules for module inclusion will be similar to package inclusion. EPIC-next modules must not replace/conflict with CentOS modules. They may use their own name-space to offer newer versions than what is offered and those modules may be removed in the next minor release if CentOS offers them then.

Build/Update Policy

Major Release

In the past, Red Hat has released a public beta before it finalizes its next major version. If possible, the rebuilders have come out with their versions of this release in order to learn what gotchas they will have when the .0 release occurs. Once the packages for the beta are built, EPIC will make a public call for packages to be released to it. Because packagers may not want to support a beta or they know that there will be other problems, these packages will NOT be auto branched from Fedora.

Minor Release

The current method CentOS uses to build a minor release is to begin rebuilding packages, patching problems and then when ready put those packages in their /cr/ directory. These are then tested for by people while updates are built and ISOs for the final minor release is done. The steps for EPIC release engineering will be the following:

1. Branch all current packages from X.Y to X.Y+1
2. Make any Bugzilla updates needed
3. Rebuild all branched packages against CR
4. File FTBFS against any packages.
5. Packagers will announce major updates to mailing list
6. Packagers will build updates against CR.
7. 2 weeks in, releng will cull any packages which are still FTBFS
   
8. 2 weeks in, releng will compose and lock the X.Y+1 release
   
9. symlinks will point to the new minor release.
   
10. 4 weeks in, releng will finish archiving off the X.Y release
    

Between Releases

Updates and new packages between releases will be pushed to the appropriate /updates/X.Y/ tree. Packagers will be encouraged to only make minor non-api breaking updates during this time. Major changes are possible, but need to follow this work flow:

1. Announce to the EPEL list that a change is required and why
2. Open a ticket to EPIC steering committee on this change
3. EPIC steering committee approves/disapproves change
4. If approved change happens but packages are in updates
5. If not approved it can be done next minor release.

Build System

Build in Fedora

Currently EPEL is built in Fedora using the Fedora Build system which integrates koji, bodhi, greenwave, other tools together. This could be still used with EPIC.

Build in CentOS

EPIC could be built in the CentOS BuildSystem (CBS) which also uses koji and has some integration to the CentOS Jenkins CI system.

Build in Cloud

Instead of using existing infrastructure, EPIC is built with newly stood up builders in Amazon or similar cloud environments. The reasoning behind this would be to see if other build systems can transition there eventually.

Definitions

Blue Sky Project

A project with a different name to help eliminate preconceptions with the existing project.

Customer

A person who pays for a service either in money, time or goods.

Consumer

Sometimes called a user. A person who is consuming the service without work put into it.

EPEL

Extra Packages for Enterprise Linux. A product name which was to be replaced years ago, but no one came up with a better one.

EPIC

Extra Packages Inter Community.

RHEL

Red Hat Enterprise Linux

Last updated 2018-05-16 19:10:17 EDT This document was imported from an adoc..

16 May 2018 11:54pm GMT

I've griped before about kernel development being scattered and spread about. A quick grep of MAINTAINERS shows over 200 git trees and even more mailing lists. Today's discussion is a partial enumeration of some common mailing lists, git trees and patchwork instances. You can certainly find some of this in the MAINTAINERS file.

• LKML. The main mailing list. This is the one everyone thinks of when they think 'kernel'. Really though, it mostly serves as an archive of everything at this point. I do not recommend e-mailing just LKML with no other lists or people. Sometimes you'll get a response but think of it more as writing to your blog that has 10 followers you've never met, 7 of which are bots. Or your twitter. There is a patchwork instance and various mail archives out there. I haven't found one I actually like as much as GMANE unfortunately. The closest corresponding git tree is the master where all releases happen.

• The stable mailing list. This is where patches go to be picked up for stable releases. The stable release have a set of rules for how patches are picked up. Most important is that the patch must be in Linus' tree before it will be applied to stable. Greg KH is the main stable maintainer. He does a fantastic job for taking care of the large number of patches that come in. In general, if a patch is properly tagged for stable yes it will show up eventually. There is a tree for his queue of patches to be applied along with stable git trees

• Linux -next. This is the closest thing to an integration tree right now. The goal is to find merge conflicts and bugs before they hit Linus' tree. All the work of merging trees is handled manually. Typically subsystem maintainers have a branch that's designated for -next which gets pulled in on a daily basis. Running -next is not usually recommended for anything more than "does this fix your problem" unless you are willing to actively report bugs. Running -next and learning how to report bugs is a great way to get involved though. There's a tree with tags per day.

• The -mm tree. This gets its name from memory management but really it's Andrew Morton's queue. Lots of odd fixes end up getting queued through here. Officially, this gets maintained with quilt. The tree for -next "mmotm" (mm of the moment) is available as a series. If you just want the memory management part of the tree, there's a tree available for that.

• Networking. netdev is the primary mailing list which covers everything from core networking infrastructure to drivers. And there's even a patchwork instance too! David Miller is the top level networking maintainer and has a tree for all your networking needs. He has a separate -next tree. One thing to keep in mind is that networking patches are sent to stable in batches and not just tagged and picked up by Greg KH. This sometimes means a larger gap between when a patch lands in Linus' branch and when it gets into a stable release.

• Fedora tree. Most of the git trees listed above are "source git/src-git" trees, meaning it's the actual source code. Fedora officially distributes everything in "pkg-git" form. If you look at the official Fedora kernel repository, you'll see it contains a bunch of patches and support files. This is similar to the -mm and -stable-queue. Josh Boyer (Fedora kernel maintainer emeritus) has some scripts to take the Fedora pkg-git and put it on kernel.org. This gets updated automatically with each build.

• DRM. This is for anything and everything related to graphics. Most everything is hosted a freedesktop.org, including the mailing list. Recently, DRM has switched to a group maintainer model (Daniel Vetter has written about some of this philosophy before). Ultimately though, all the patches will come through the main DRM git repo. There's a DRM -tip for -next like testing of all the latest graphics work. Graphics maintainers may occasionally request you test that tree if you have graphics problems. There's also a patchwork instance.

16 May 2018 6:00pm GMT

We're about to finish the three days long first GNOME Performance Hackfest here in Cambridge.

We started covering a few topics, there are three major areas we've covered and in each one of those there has been a bunch of initiatives.

GNOME Shell performance

Jonas Adahl, Marco Trevisan, Eric Anholt, Emmanuele Bassi, Carlos Garnacho and Daniel Stone have been flocking together around Shell performance. There has been some high level discussions about the pipeline, Clutter, Cogl, cairo/gtk3 and gtk4.

The main effort has been around creating probes across the stack to help Christian Hergert with sysprof (in drm, mutter, gjs…) so that we can actually measure performance bottlenecks at different levels and pinpoint culprits.

We've been also looking at the story behind search providers and see if we can rearchitect things a bit to have less roundtrips and avoid persistent session daemons to achieve the same results. Discussions are still ongoing on that front.

GNOME Session resource consumption

Hans de Goede put together a summary of the resource consumed in a typical GNOME session in Fedora and tweaks to avoid those, you can check the list in the agenda.

There are some issues specific to Fedora there, but the biggest improvement that we can achieve is shutting down's GDM's own gnome-shell instance, for which Hans already has a working patch. This should reduce resource consumption by 280megs of RAM.

The second biggest target is GNOME Software, which we keep running primarily for the shell provider. Richard Hughes was here yesterday and is already working on a solution for this.

We are also looking into the different GNOME Settings Daemon processes and trying to figure out which ones we can shut down until needed.

Surely there's stuff I've missed, and hopefully we'll see blogposts and patches surfacing soon after we wrap up the event. Hopefully we can follow up during GUADEC and start showing the results.

On Tuesday we enjoyed some drinks out kindly hosted by Collabora.

I'd like to thank the Eben Upton and the Raspberry Pi Foundation for sponsoring the venue and sending Eric Anholt over.

16 May 2018 3:04pm GMT

You visited the Fedora table at SouthEast LinuxFest (SELF) 2018!

16 May 2018 2:18pm GMT

You organized a party for the release of Fedora 28

16 May 2018 2:09pm GMT

Attended a local release party to celebrate the launch of Fedora 28!

16 May 2018 1:50pm GMT

You visited the Fedora booth at OSCAL 2018 in Tirana, Albania!

16 May 2018 1:42pm GMT

Cockpit is the modern Linux admin interface. We release regularly. Here are the release notes from version 168.

Improve checks for root privilege availability

Many actions, like joining an IPA domain or rebooting, can only be performed by an administrator (root user).

Cockpit previously checked if the user was a member of the wheel (used in Red Hat Enterprise Linux, CentOS, and Fedora) or sudo (used in Debian and Ubuntu) groups to enable these actions. Simple group checking was insufficient, as different group names are used by other operating systems and configurations, or a system might be set up to rely on custom user-based sudo rules.

Cockpit no longer makes assumptions about special groups. Now, Cockpit simply checks if the user is capable of running commands as root.

As a result of this fix, privileged operations now work by default with FreeIPA, which uses the admins@domain.name group.

Try it out

Cockpit 168 is available now:

• For your Linux system
• Source Tarball
• Fedora 28

16 May 2018 11:00am GMT

You visited the Fedora booth at FLISOL 2018!

16 May 2018 9:17am GMT

All the current Flatpak runtimes in wide use are based on the 1.6 Freedesktop runtime. This is a two-layered beast where the lower layer is built using Yocto and the upper layer is built using flatpak-builder.

Yocto let us get a basic system bootstrapped, but it is not really a great match for the needs of a Flatpak runtime. We've long wanted a base that targeted sandboxed builds which is closer to upstream and without the embedded cross-compiled legacy of Yocto. This is part of the reason why Tristan started working on BuildStream, and why the 1.8 Freedesktop runtime was created from scratch, using it.

After a herculean effort from the people at Codethink (who sponsor this effort) we now have a working Yocto-free version of the Platform and Sdk. You can download the unstable version from here and start playing with it. It is not yet frozen or API/ABI stable, but its getting there.

The next step in this effort is to test it as widely as possible to catch any issues before the release is frozen. In order to do this I rebased the Gnome nightly runtime builds on top of the new Freedesktop version this week. This is a good match for a test release, because it has no real ABI requirements (being rebuilt with the apps daily), yet gets a fair amount of testing.

WARNING: During the initial phase it is likely that there will be problems. Please test your apps extra carefully and report all the issues you find.

In the future, the goal is to also convert the Gnome runtimes to BuildStream. Work on this has started, but for now we want to focus on getting the base runtime stable.

16 May 2018 8:03am GMT

Cela fait depuis Fedora 19 que je publie sur la liste de diffusion de Fedora-fr une revue de presse de chaque sortie d'une nouvelle version. Récapituler quels sites en parle et comment. Je le fais toujours deux semaines après la publication (pour que tout le monde ait le temps d'en parler). Maintenant, place à Fedora 28 !

Bien entendu je passe sous silence mon blog et le forum de fedora-fr.

Sites web d'actualité

• Linuxfr ;
• NextINpact ;
• Développez.com ;
• Programmez! ;
• Toolinux ;
• QuebecOS ;
• Génération NT.

Soit 7 sites sur les 25 contactés.

Blogs, sites persos ou sites non contactés

• Guillaume Kulakowski ;
• Frédéric Bézies.

Soit 2 sites.

Bilan

Le nombre de sites parlant de Fedora 28 est en légère baisse, deux blogs en moins. Beaucoup d'articles se fondent sur ce que j'ai moi même rédigé (que ce soit la version courte ou longue). Notons que developpez.net m'a contacté pour publier la version longue les prochaines fois, ce que je vais essayer de faire bien entendu.

La semaine de sa sortie, nous avons eu une augmentation de visites par rapport à la semaine d'avant de cet ordre là :

• Forums : 4% (soit 250 visites environ)
• Documentation : baisse de 1% (soit environ 40 visites)
• Le site Fedora-fr : 43% (soit 500 visites en plus)
• Borsalinux-fr : 344% (soit 90 visites en plus)

À tenir compte de la situation particulière avec une sortie le 1er mai qui est un jour férié globalement dans l'espace francophone, hors Québec.

Si vous avez connaissance d'un autre lien, n'hésitez pas à partager ! Rendez-vous pour Fedora 29.

15 May 2018 3:00pm GMT

A critical security vulnerability was discovered and disclosed earlier today in dhcp-client. This DHCP flaw carries a high risk to your system and data, especially if you use untrusted networks such as a WiFi access point you don't own. Read more here for how to protect your Fedora system.

Dynamic Host Control Protocol (DHCP) allows your system to get configuration from a network it joins. Your system will make a request for DHCP data, and typically a server such as a router answers. The server provides the necessary data for your system to configure itself. This is how, for instance, your system configures itself properly for networking when it joins a wireless network.

However, an attacker on the local network may be able to exploit this vulnerability. Using a flaw in a dhcp-client script that runs under NetworkManager, the attacker may be able to run arbitrary commands with root privileges on your system. This DHCP flaw puts your system and your data at high risk. The flaw has been assigned CVE-2018-1111 and has a Bugzilla tracking bug.

Guarding against this DHCP flaw

New dhcp packages contain fixes for Fedora 26, 27, and 28, as well as Rawhide. The maintainers have submitted these updates to the updates-testing repositories. They should show up in stable repos within a day or so of this post for most users. The desired packages are:

• Fedora 26: dhcp-4.3.5-11.fc26
• Fedora 27: dhcp-4.3.6-10.fc27
• Fedora 28: dhcp-4.3.6-20.fc28
• Rawhide: dhcp-4.3.6-21.fc29

Updating a stable Fedora system

To update immediately on a stable Fedora release, use this command with sudo. Type your password at the prompt, if necessary:

sudo dnf --refresh --enablerepo=updates-testing update dhcp-client

Later, use the standard stable repos to update. To update your Fedora system from the stable repos, use this command:

sudo dnf --refresh update dhcp-client

Updating a Rawhide system

If your system is on Rawhide, use these commands to download and update the packages immediately:

mkdir dhcp && cd dhcp
koji download-build --arch={x86_64,noarch} dhcp-4.3.6-21.fc29
sudo dnf update ./dhcp-*.rpm

After the nightly Rawhide compose, simply run sudo dnf update to get the update.

Fedora Atomic Host

The fixes for Fedora Atomic Host are in ostree version 28.20180515.1. To get the update, run this command:

atomic host upgrade -r

This command reboots your system to apply the upgrade.

Photo by Markus Spiske on Unsplash.

15 May 2018 2:59pm GMT