Articol despre Grid Scris de O'Reilly

The State of the World in Grid Computing

Until the term "grid computing" hit the scene, there were basically two ways to make a distributed, parallel application, and they both involved a lot of infrastructure development and a lot of programming.

On the infrastructure side, one generally had to design a more-or-less custom network for running distributed applications that was optimized for moving the inter-process communications of your application efficiently. Unless you're solving toy problems, this involves lots of expensive networking hardware and a lot of planning.

On the programming side, unless you were really brave and wanted to invent your own mechanisms for slicing and dicing data and getting it to processors, you used either the Message Passing Interface (MPI) or the Parallel Virtual Machine (PVM) systems as the communications libraries to effect the parallel communications in your application.

In the last six months there have been several major announcements of efforts to formalize the grid computing revolution. IBM and Sun have each announced hardware systems specially designed to support grid computing. They’ve also delivered software packages to make running programs in a distributed environment easier and more productive, with software libraries which allow everything from Java to Legacy Mainframe applications join the grid.

These join a number of existing efforts, including those sponsored by the Defense Advanced Research Projects Agency (DARPA) and the National Science Foundation (NSF). They include the Cactus and Globus projects that provide users with an entire set of libraries and communications systems to grid-enable their applications.

These are great projects that solve a lot of the low-level networking issues and allow many kinds of applications to take advantage of distributed-computing capabilities. The only drawback is the large investment of time and resources to convert an existing application to run on the grid. And once you port your application to a "grid" structure there’s no guarantee that you’ll get better or faster results, which should be one of the primary goals of creating a distributed or parallel application.

So, with all of the buzz and hype, what can you (and your enterprise) actually do right now to speed up your applications and take advantage of this new grid-computing paradigm?

The Powerllel System

One of the hottest new systems to help businesses jumpstart the process of parallelizing applications and reaping the benefits of faster applications has been developed by a New York City-based start-up called Powerllel.

The founders of Powerllel have a long history at large brokerage firms of developing trading analytics, a process that takes the idea of needing results "fast" to a whole new level. In the financial markets there aren’t many differentiators -- when you get down to it everyone is using the same algorithms, and more or less the same data. What differentiates the winners from the losers in trading systems is how fast and accurately you can get to a result and how much data you can plow through in the time available to make a trade.

For many firms the answer has been to buy traditional super-computers, but this is an expensive and limited solution because of the expense of these systems and the insatiable appetite for computer power that financial (and other businesses) have.

The Powerllel developers figured there had to be a way to capitalize on off-the-shelf hardware (personal computers, workstations, and UNIX servers) to make parallel-application development easier and more effective, while ensuring that existing investment in software technologies (like applications written with MPI and PVM and custom parallel libraries) didn’t go to waste. They accomplished this and a whole lot more with the Powerllel Software Suite.

The 2002 O'Reilly Emerging Technologies Conference explored how P2P and Web services are coming together in a new Internet operating system.

DNET, Lobsters and Adapters: Oh, MY!

Powerllel has created an innovative system that allows applications to be developed and/or re-targeted in a matter of days, as opposed to months (or even years), that is typical with most parallel program-development processes.

The Powerllel system consists of three major components:

• A distributed network layer, called DNET, that completely hides the underlying networking and communication system from an application. This means programmers no longer have to code their applications to a particular network communications model (MPI, PVM, Linda, etc.). The DNET system decides what the most effective communications mechanism is, given the computational resources available on the network and what communications models they support.

• A load-balancing system called Lobster (Load-Balancing Sub-Task ExecuteR). Lobster is an integrated mechanism to distribute slices of a computation across a network of computational resources (such as a collection of Linux boxes, Suns, mainframes, whatever) and to ensure that failover, recovery, and other critical services are managed on behalf of the application.

• Adapters -- application frameworks that represent various styles of execution for distributed/parallel computations. These adapters include simple parametric systems (send a bunch of parameters to a subroutine, get back a discrete result, much like SETI@home-type calculations), and complex tree-structured recursive systems where all of the computations are interdependent and intertwined (typical of financial models, pharmaceutical modeling, weather modeling, and other real-world problems).

All of this is brought together in a system that allows the developer to focus on the problem being solved, rather than the intricacies of how to structure communications between nodes or whether MPI is a better model than PVM for a given class of problem.

What's the Big Deal?

Traditional distributed and/or parallel applications come in two basic flavors: "embarrassingly parallel" and "deeply parallel."

Embarrassingly parallel problems are things like SETI@home, where you have a whole lot of data that you’re applying some kind of analysis to, but it doesn't really mater where you break up the data. In effect, you're handing a bunch of parameters (the data) to a subroutine and waiting for some discrete set of results to come back. More importantly, the results from one data block are not linked to the results from any other block. Everything is discrete; this is often referred to as parametric computing.

Deeply parallel problems are those in which most, if not all, of the calculations are intertwined with some other part of the computational process, either as an input or a potential side effect. These are often referred to as non-linear or tree-structured problems. Weather modeling, financial models, and drug development are great examples: every aspect of the computational process is interlinked.

These deeply parallel problems can take vast amounts of time to get right if coded by hand: there are just too many things that have to be done exactly right, from networking decisions and communications libraries to the decomposition of the algorithms themselves. Mistakes can costs millions in redesign, re-coding, and delays in deployment.

The Powerllel solution is a revolution in the development of "grid" applications because the Powerllel Software Suite presents a system to the developer where they provide the fundamental algorithm to be solved, compile the application, and they're good to go across any range and type of computing system (Windows, Mac, Linux, UNIX, etc.). The DNET and Lobster mechanisms make all the hard decisions about network topology, routing and failover/robustness, while the Adapters determine which kind of processing structure makes sense given the kind of algorithm being solved.

The Bottom Line

Grid computing is the newest and hottest trend in high-performance computing, and will help companies -- eventually -- take advantage of the power of distributed systems. What's been missing is the ability to simplify the application development or porting process and get applications up and running without months and/or years of expensive error-prone development: Powerlell's Software Suite just may be the killer-app for grid computing.

CERN

cel mai mare centru de cercetare in fizica particulelor din lume

            CERN este Laboratorul European pentru Fizica Particulelor Elementare, cel mai mare centru de cercetare in fizica particulelor din lume, situat in partea de nord-vest a Genevei la granita dintre Franta si Elvetia. Conventia in baza careia s-a infiintat CERN-ul a fost semnata la 29 septembrie 1954. De la initial 12 state semnatare ale Conventiei, la ora actuala CERN cuprinde 20 de state membre. In timp CERN-ul a devenit un exemplu de colaborare internationala, cunoscut de asemenea si ca locul unde a fost creat web-ul.

Statele membre fondatoare CERN au fost: Belgia, Danemarca, Germania (pe atunci Germania de Vest), Franta, Grecia, Italia, Norvegia, Suedia, Elvetia,  Olanda, Marea Britanie, Iugoslavia. Ulterior au aderat urmatoarele tari: Austria (1959), Spania (1961), Portugalia (1985), Finlanda (1991), Polonia (1991), Ungaria (1992), Republica Ceha (1993), Slovacia (1993), Bulgaria (1999). Iugoslavia s-a retras in 1961. Alte 8 entitati (organizatii internationale sau tari) au statut de observator: Comisia Europeana, India, Israel, Japonia, Rusia, Turcia, UNESCO, SUA.

            CERN este laboratorul unde oamenii de stiinta isi unesc eforturile pentru a studia structura elementara a materiei si fortele care o guverneaza. Aproape 7000 de oameni de stiinta, jumatate din fizicienii lumii care lucreaza in domeniul particulelor elementare, utilizeaza facilitatile CERN. Ei reprezinta 500 de universitati si peste 80 de nationalitati. Activitatea CERN este structurata pe trei domenii, toate programele dezvoltate vizand teme care constituie prioritati stiintifice mondiale: 

●         cercetare fundamentala – la CERN se incearca sa se raspunda la cele mai fundamentale intrebari despre natura: Ce este materia? Cum apare aceasta? Cum se grupeaza in obiecte complicate precum stele, planete, oameni?

●         cercetare aplicativa si dezvoltare tehnologica – desi CERN-ul inseamna cercetare fundamentala, Laboratorul joaca un rol vital in dezvoltarea tehnologiilor de maine. De la stiinta materialelor la calculatoare, fizica particulelor elementare are nevoie de cea mai inalta performanta, transformand CERN-ul intr-un important “banc de probe” pentru industrie.

●         formarea viitorilor specialisti – CERN joaca un rol important in educatia tehnologica avansata. O gama larga de forme de pregatire si specializare atrag multi tineri cercetatori si ingineri la CERN. Majoritatea isi continua cariera in industrie, unde experienta lucrului intr-un climat “high-tech” multinational este deosebit de valoroasa.

 In cadrul experimentelor de la CERN au fost obtinute realizari remarcabile, printre care:

·       1973: Descoperirea curentilor neutri in camera cu bule Gargamelle

·       1983: Descoperirea bosonilor W si Z in cadrul experimentelor UA1 si UA2

·       1995: Obtinerea pentru prima data a antimateriei in cadrul experimentului PS210

·       2001: Punerea in evidenta a violarii simetriei CP in cadrul experimentelor NA48

In 1984 a fost acordat Premiul Nobel pentru fizica lui Carlo Rubbia si Simon van der Meer pentru descoperirea bosonilor W si Z.

In 1992 a fost acordat Premiul Nobel pentru fizica lui Georges Charpak pentru inventia si dezvoltarea detectorilor de particule, in special pentru “multiwire proportional chamber."

            In momentul de fata la CERN se construieste cel mai performant super-accelerator de particule din lume - Large Hadron Collider (LHC). Acest accelerator este instalat intr-un tunel cu circumferinta de 27 de km, unde a activat un accelerator mai vechi - Large Electron Positron collider (LEP). Studiind ciocnirile la energii foarte inalte, fizicienii care lucreaza la LHC vor intelege mai bine structura Universului si originea lui. Acest accelerator va genera cantitati foarte mari de date experimentale, pe care CERN-ul le va directiona catre laboratoare din intreaga lume pentru procesare distribuita (tehnologia GRID). 

            Colaborarea Romaniei cu CERN:

            Scurta istorie:

●         1991 -  a fost semnat primul Acord, reprezentand cadrul necesar participarii cercetarii                       si industriei romanesti la programele CERN

●         2001 -  Ministrul Afacerilor Externe si Ministrul Educatiei si Cercatarii viziteaza CERN

●         2002 -  a fost semnat Acordul de cooperare intre Guvernul Romaniei si CERN referitor          la dezvoltarea viitoare a cooperarii stiintifice si tehnice in cadrul proiectelor de                cercetare ale CERN, acord inca in vigoare

●         2003 -  Presedintele Roamaniei viziteaza CERN

●         2004 -  infiintarea Comitetului National Romania – CERN, organ consultativ cu scopul                       dezvoltarii colaborarii dintre institutiile de cercetare romanesti si CERN

●         2006-   a fost semnat Memorandumul de Intelegere vizand colaborarea pentru                                     desfasurarea si exploatarea  Worldwide LHC Computing Grid, in cadrul careia           Romania participa cu un centru           Tier 2

            Peste 70 de fizicieni si ingineri romani lucreaza in cadrul proiectelor CERN. Institutul National de Fizica si Inginerie Nucleara Horia Hulubei participa in mod oficial la trei experimente LHC (ATLAS, ALICE and LHC-b), la construirea detectorilor si la pregatirea analizei datelor.  Un grup de cercetatori si studenti de la Universitatea “Politehnica” din Bucuresti este implicat in dezvoltarile de avangarda in tehnologia informatiei de la CERN. Institutii stiintifice romanesti de renume, precum Institutul de Stiinte Spatiale si Universitatea din Bucuresti, contribuie de asemenea la experimentele de fizica fundamentala de la CERN. In plus, CERN-ul constituie un centru de fuziune si de initiere a colaborarii intre comunitatea stiintifica romaneasca din tara si cea din strainatate.