Distributed Raytracing Proposal

Distributed Raytracing Proposal

Post by Eric Jorgens » Thu, 12 Oct 1995 04:00:00



I'm not sure how much people are thinking or working on this subject, so
I decided to take it to the usenet to toss the idea around.

It appears to me that if I want to do some distributed ray tracing, I have
to have some proprietary software and a local network. There does not seem
to be any sort of distributed raytracing solution for the average user
(like myself) who cannot afford to fork out bucks for the hardware and
software.

I think it would be beneficial if we as a community came up with a
free protocol for distributed raytracing over the internet.   I'm posting
the hopes that I can generate discussion and collect ideas for this
protocol.  I'm not aware of any work in this area and I think I see a
good opportunity to generate a good standard before there is too much
competition.

To fuel the discussion, I am including a Rev 0 document I drafted to
sketch out some of the details for my ideas.  I am very interested an any
feedback anyone may have.

-e

--------------- My WEB page: http://www.rt66.com/smeagol/ --------------------
---- Ground Up: http://www.rt66.com/smeagol/html/groundup/groundup.html ------

        DRIP Specification Rev 0.0

Distributed Raytracing Internet Protocol
        - Eric Jorgensen (1995)
          smea...@rt66.com

==== INTRODUCTION

        Anybody who has dabbled in raytracing to any extent is well aware
that it is a time consuming, CPU intensive process.  Distributing the
computational load over many CPU's is a tried and true method of cutting
the time to render an image.

        The computer revolution is now filling the world with computers,
which are largely idle nearly all of the time.  Thanks to the growth of
the internet, many of these computers also happen to be linked. This
presents an interesting opportunity to make better use of this vast
amount of wasted CPU cycles.

        This document exists to define a protocol for Distributing
Raytracing jobs across CPU's linked through the internet.

==== SCOPE

        The scope of this document is not yet defined.  On the outset, the
scope shall only be to define a protocol to allow computers to share
information about raytracing.  If the need and momentum present
themselves, this document could be expanded to include a scene and
animation descriptor protocol, but this is currently a long range goal in
the author's mind.

==== IDEAS

        The specs are not fully developed, so I thought I would provide
some of my thoughts on the topic first.

        - The basic idea behind DRIP is to allow specific raytracing
          packages to query the internet for available CPU's and
          utilize them for rendering scenes or animations.

        - DRIP would be most useful as a platform independent
          protocol and it should also be designed to prevent the
          distribution of viruses.  Therefore, the protocol only
          outlines how to distribute data.  Executables are not
          to be distributed.  Computers that allow drip access must
          have rendering software available locally.

        - DRIP participation is voluntary.  The idea here is that there
          is a central DRIP list that has all of the participating
          computers and their IP addresses.  In order to be a member
          of the drip club, you must submit the name of your computer to
          the list.

        - Individual computers should have the right to refuse drip
          access at any time, even in the middle of rendering.  

        - To encourage fairness and efficiency, a commercial provider
          may want to maintain a database of DRIP computers and keep
          information on utilization.

        - The DRIP list will have the ability to maintain schedules so
          that a participant can give times when their computer is
          most likely to be idle.

        - As the intricacies of eMoney are ironed out, it might
          be prudent to a reimbursement specification.  ie: a person
          may want to give preference to renderers who are willing to
          pay a nominal fee for CPU usage.  

          I have no idea what one would charge for such a service.  I am
          thinking that people would charge for CPU cycles rather to
          prevent cheating.  In this case, the protocol might need to be
          able to define how many calculations a rendering needs.

        - The protocol should allow for the specifacation of  partial
          renderings.

==== Specifications
0.0  Revision History

  10/11/95 Rev 0.0    Initial document.  (Eric Jorgensen)

1.0  Software

  1.1  DRIP requires the presence of a daemon process that is capable
       of sending and receiving DRIP packets (DROPS?).  The daemon should
       also be aware of rendering software available locally and have
       the ability to run them.

  1.2  DRIP requires the presence of at least one rendering software
       package.  (If the scope of DRIP ever includes scene descriptors,
       this package will have to be DRIP compliant as well.)

2.0  Communications

   (DRIP will require some standard internet protocol to communicate,
    like ftp, talk, telnet, or perhaps even email and IRC.  I need to
    collect more info before I decide what goes here.)

3.0  The DRIP list

  3.1  It is assumed that any DRIP participant will be added to the DRIP
       list.

  3.2   Drip lists will be maintained through the public domain, much like
        URL's.  

  3.3  The list will follow this line-based format:

        [name],[IP address],[CPU rating],[package codes],[Idle schedule]

        3.31 Name- It is assumed this will be the hostname, but it can
             be anything since it is not critical.

        3.32 IP address- Self explanatory.

        3.33 CPU rating- We will need some iEEE floating point rating
             standard here.  Perhaps it will be necessary to define
             one specifically for raytracers.

        3.34 Package Codes- 5 character codes to indicate rendering
             software available and major version number.  The first
             three characters indicate the package and the last two
             are for the version.  No spaces are allowed and the names
             must be period delimited. All packages must be registered in
             this document.  Current packages defined are:

                Persistence of vision  
                        POV01,POV02,(POV03)

                (I need some input here)

        3.35 Idle schedule-  This last field continues to end of line
             and contains space-delimited codes which define the idle
             schedule.  The codes are as follows:

                [DAYCODE].[TIME].[RANGE]
                        DayCodes are: MON,TUE,WED,THU,FRI,SAT,SUN,ALL
                        Time format (24 hr time): HH:MM
                        Range Format: HH:MM

                (More to come)

              Since DRIP lists are maintained locally, times should be
              converted when received to apply to local standard time.
              ie: The computer may be in Finland, so make sure you get
              the times right.    

4.0  DRIP packets

        4.1 DRIP packets are the standard form of communication between
            DRIP clients.  A DRIP conversation proceeds like this.

                1) DRIP sender sends a DRIP query to a receiver with this
                   information:
                        - Sender's Name and IP address
                        - Sender's CPU rating
                        - Sender's Rendering Packages available
                        - Sender's Idle schedule
                        - Number of FP calculations desired
                        - memory size of image desired
                        - Memory required for scene description
                        - Rendering Package desired
                        - Image format and color depth desired
                2) Receiver sends this information back:
                        - Receiver's Current Availability (with comment)
                        - Receiver's Current CPU rating
                        - Receiver's Rendering Packages available
                        - Receiver's Current Idle Schedule
                        - FP calculations offered
                        - Image format and color depth available
                3) Sender replies with:
                        - Go or no go on render
                        - Package to use
                        - Number of FP calculations
                        - Image format and color depth request
                4) Receiver Accepts or rejects the sender's reply
                5) Sender transmits scene data
                6) Receiver ACK's
                7) Receiver renders, then transmits a DONE packet
                8) Sender ACK's
                9) Receiver sends back image in format of it's choosing.
                        - In general, lossy compression should not be
                          used unless requested.
                10) Send ACK's with RECEIVED and DONE message.          

        4.2 Actual DRIP code will be defined later as the procedure is
            ironed out.  

        4.3  DRIP Termination- A protocol will go here to indicate how
             a receiver can terminate a rendering so that data is
             preserved and the sender knows about the termination.  

--
--------------- My WEB page: http://www.rt66.com/smeagol/ --------------------
---- Ground Up: http://www.rt66.com/smeagol/html/groundup/groundup.html ------

 
 
 

1. Proposal for new lock ownership scheme to support NFS over distributed filesystems

Hello all,

As a prelude to adding support in kNFS(lockd) to pass lock information via
the VFS interface I want to firsr propose some minor changes that will
enable us to have a single lock ownership space that is shared and enforced
at all nodes.

Currently lockd identified range locks with two fields fl_owner and fl_pid.
NFS code places a memory pointer (pointer to host structure representing
NFS client holding the lock) in fl_owner which works great in single node
NFS but breaks when we attempt to pass lock requests to an the underlying
distributed filesystem in the context of a clustered NAS head. This can be
easily fixed by filling fl_owner field with the IP address of the NFS
client requesting the lock, which is just a minor patch.

The patch above, however, does not fix all our problems as locks acquired
locally in the NFS server nodes could collide with NFS locks. Currently,
the fl_owner field for locks acquired via the local file system is filled
with another memory pointer (i.e. current->files). This also breaks appart
when using a distributed file system. My proposal for this would be use a
local IP address to put in the fl_owner field of locks acquired locally in
a node.

These two patches cause no problem for current single node NFS servers and
enable Linux as a platform to support distributed filesystems and NFS
serving simultaneously and correctly.

I would like to get your feedback on this proposal and if accepted as it is
I will later follow with patches for current versions of the kernel NFS.

Regards, Juan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. vlock version 0.6

3. raytrace

4. Juke-Box On AIC7xxx

5. Good distribution for raytracing?

6. Logitec Busmice and X

7. Raytracing SCED pov-files?

8. UDMA-Modes with Kernel 2.2.5

9. multiple file output for frames of animation with raytrace

10. Raytracing and Linux

11. Silly_Rabbit on RayTracing

12. RAYTRACE

13. Linux for raytracing