Very Interesting

May 7, 2012 at 1:49 AM

I found this project via http://jetfire.codeplex.com and it looks very interesting. I already have issue managing complex state and access in a system i have been writing and this looks like it could solve my issues from the code i have been looking at.

I love the idea of creating dynamic workflows etc from strings so that i can build screens into my management webpages allowing the editing and changes to my workflows without code changes requiring the whole application to be recompiled. Brilliant idea and well done!

The access control and the different options to choose from also seem to be very well thought out.

 

I just need to work out how to integrate this all into my current code and database/objects. I assume mapping to and from the DB and seralizing and deserializing is something i need to do myself.  Are there any good examples of how to integrate this workflow in existing apps with DB storage etc?  Just a generic basic example just to get an idea of how you recommend things fit together?  It seems i need to do something in Server Operations to make it persist data but since it has not methods and does nto extend any other classes i am not sure how it is meant to work.

Coordinator
May 7, 2012 at 6:28 PM
Edited May 7, 2012 at 6:32 PM

Thank you for your interest and your kind comments.

For simple session style persistence (save and restore everything) you can use the JSON serialization code found in the 'Jazz.Json' project (using Newtonsoft JSON.Net as a building block).  There is a test case file, "5.Serialization.cs", demonstrating session persistence which can be found in the 'JazzClientTest' project.   This all part of the Source Code download (get the latest download on the "SourceCode" tab page).

The current serialization technique converts Jazz objects into individual JSON fragments (one fragment for each Jazz object).  The fragments can be stored by their ID (Guid).  Other objects are stored by value within the fragments - which is okay for most objects.  If you need to store/serialize an object by reference one solution would be to wrap that object in Jazz object.  You may need to create a custom serialization converters for some of your objects using JSON.Net.  

The Client Nexus will shortly support references to offline, and unavailable Jazz objects, when the ACL features are added.  This will support, among other things, dynamic loading of Jazz objects from offline storage.

Regarding persistence to a DB (to provide searching) that is something we are just looking at.   We have a model that we have used in the Jetfire project using a software server (Server Nexus) to coordinate activity of multiple clients (Client Nexus).   It uses a standard Microsoft SQL DB server. Our current thoughts is to use a 'no-sql' server for the back end.  Any feedback would be most appreciated.

I hope this helps.

regards,
John Hansen 

May 7, 2012 at 9:52 PM

Hi John,

I have had a small play (very small play) so far but the only thing i did not really like was that PostSharp reference. Just a pain as you need to register each developer with them etc. Are there no better alternatives on Nuget? It just would have been nice if it was a dll already included that does not require install and full extension just to be run jazz as part of a bigger system and comes with quite a few license restrictions. If it is the only one that does the job then that is that but would be nice to use something less restrictive if possible.

 

As to the database personally i would make it a simple interface if possible with a few basic implementations people can either use or use as a sample. If storing json the you are right some of the json non sql databases may be ideal or even a normal file but there are also systems that will need it in a sql database and may even need control over how it is stored and just uses your implementation as a base. Also note i am saying SQL not MS SQL as MS SQL locks people to windows systems and generally more expensive hosting.

 

Thanks for your answers above it gives me a much better idea on how i will make this all fit together.

With profiles did you seem them been sort of like a group where it contains multiple roles and you add multiple users to that profile or been custom per user? I assume roles are your fine grain control then you would group a bunch of roles together for say and Admin, Manager or staff and generally all Admins would have the same roles etc. Is that what you envisioned or did i get the wrong idea of what profiles are?

Regards,

Chris


Coordinator
May 8, 2012 at 12:02 AM

We are working to eliminate the need to download Postsharp, but don't have a solution yet.  Postsharp does appear to be the only viable candidate for this application.

With respect to databases we are sensitive to all the points you raised.  Well endeavor to provide flexibility and alternatives our storage solutions.  When we do come out with a server solution will definitely make a memory version to support testing and session demos. 

With users, profiles and roles I think we are on the same page.   We are getting the documentation together first by writing good comments and example programs in the form of test cases.  Shortly we should be able to show some User and Profile classes that themselves employ roles, states, etc to control modification. In the meantime here are some more information that may help.

In Jazz everything is a Jazz object including Users, Profiles and Roles.   That means all these objects can be inherited and you can make your own user, profile or role classes while retaining their first class functionality within Jazz.   An inherited User class makes sense because a developer would probably what to add more contact information such Last Name, address, etc.  The inherited user class can also be a full workflow thus giving flexibility on how and who can update a user instance.

Users, in Jazz, are essentially a collection of profiles.  Profiles are a collection of roles with a link to a single user.  When a user logs in to a client nexus, it is a single profile that logs in to the client nexus.  This allows for a single user to have multiple profiles; however only a single profile is active on a single client nexus at a time.  For example a user called 'Joe' could have a profile called 'Plumber' and another called "Soccer Coach".  This way only some actions, and data, may be available for one profile and other data and actions for another profile.

Currently users (profiles) only impact which methods and properties are executable or readable/writable.  With the introduction of ACL the visibility of entire Jazz objects or workspaces (directory structure for Jazz objects) can be controlled.   One notable feature, using ACL, is the ability to make a Jazz object read only for one user(profile), fully accessible to another user and not accessible at all to a third user.  Enforcing access rules would not just be the client nexus, but also the server nexus.

ACL is one of the next key priorities for the Jazz project.

regards,

John

May 8, 2012 at 4:13 AM

Thanks John that makes a lot of sense and explains your thinking very well. I was thinking profiles were slightly different to what they are.  Can i suggest you consider another structure that contains multiple related Roles grouped together that can be assigned to the profile of multiple people. The reason i suggest this is in my case i was thinking to have a large number of fine grain roles to control access but then put them in groups for common tasks user may perform. A role such as Admin is very broad in what they should have access to and i would more see that as a role group containing roles like AccountViewer, AccountModifier, etc where a User many just have AccountViewer rights say.  It is a greatly simplified example but basically forming starndard groups that give access to specific tasks and some groups that have access to almost all tasks.  

Similar to how permission groups etc work in windows now so that i can change access just by changing the roles or role groups instead of changing code to be limited or not limited to new role.  The idea of a role of say Admin seems to be bad as almost every method would have that group attribute on it as well as any other high level roles where as if you had fine grained roles of say ViewAccount then you can just add it or remove it from a role group without changing code.

Since i am still getting my head around your system that could be a very bad idea but i would like to understand why if something like that is a bad idea or maybe it is something i can do myself by extending roles to be able to child roles or something.  That could be an even better way to do it.

Also i am not sure what features of postsharp you are using but http://stackoverflow.com/questions/5622024/opensource-alternative-to-postsharp-that-supports-easy-custom-attributes has a number of alternative init though none seems to meet what the owner needed,

Regards,

 

Chris

 

May 8, 2012 at 9:25 AM

Is there a interface or something i can implement in my classes to make my existing classes a Jazz object but enable serialisation and mapping code to access the state property so i can use my nhibernate objects as jazz objects but have nibernate save just the extra state info Jazz needs to another column as a property or something. Just thinking of different options for integrating with my existing system.

Coordinator
May 8, 2012 at 12:07 PM
Edited May 8, 2012 at 1:19 PM

Regarding serialization in Jazz there are 2 builtin options.  Both, I think, are quite flexible.  There are totally optional as they are in a separate project, 'Jazz.Json'

There is a small amount of support code in the 'Jazz' core project ('Serialization Services') to handle object creation (does not execute the constructor) and forward/circular references in Jazz objects.  This is generic code and can be used by any serialization process.

The 2 serialization mechanisms are as follows:

1) JazzSerializer - This basically wraps Json.Net (Newtonsoft) and uses Jazz's core 'Serialization Services'.   Currently in behaves much like a traditional serializer; however it handles object inheritance and forward references.  It is further described in an above post in this thread.

2)The namespace 'TrackerRealm.Jazz.JsonSpecialized' contains a second option that only serializes and deserializes the base Jazz object (the 9-10 fields that make up a jazz object).   This uses Jazz's core 'Serialization Services' so that forward references can handled seamlessly.   I think this may be what you are requesting.   Note: This feature is included in the source code; however it is currently in development.

May 8, 2012 at 1:08 PM

Thanks i think the second option is what i want.  Basically i already have code that maps my objects to DB etc so i just want to get the extra jazz info that gets added to my existing objects if i make the jazz objects so that my current system can save that info along with the current info it saves for the object.  I will have a look at the code.

Coordinator
May 8, 2012 at 1:16 PM
Edited May 8, 2012 at 1:18 PM

Regarding your 11:13 post above (although it is only 11:13 in my time zone).

1. - I like your ideas and comments.  They are very much appreciated.

2. Regarding collection of roles, this is something you can do in the current version of Jazz with the 'GrantProperty' feature.  See the file "4.GrantPropertyTest.cs" in the download for an example.

3. Regarding a collection of roles object.  Yes that is something we can and should implement.   As you suggest it probably the easiest way is to build it into the existing jRole object - ie it can store a collection of roles with the initial size of 1.  Make the collection a size of 0 would disable the role object.  The class can be inherited to add features such modifying the collection of roles a function of the state of the 'Role' object.  Making the collection truly dynamic, such as a function of time of day, gets a bit trickier.

I would add a few restrictions on role collections, initially anyway, to prevent cascading issues.

4. Thank you for your Postsharp alternative.  I will check it out.

Coordinator
May 8, 2012 at 1:23 PM
Edited May 9, 2012 at 8:00 PM

Regarding Serialization - Second option - I will make it a priority to finish off that code and provide some usage example code.

May 9/2012 - Update:  Changeset 'adcf0beb397d' adds a 'Populate jazz object from JSON text' method.

May 8, 2012 at 1:25 PM
JohnHansen wrote:

Regarding Serialization - Second option - I will make it a priority to finish off that code and provide some usage example code.

Thanks. I will also play around with it and do some tests to better understand what it can do for me and how i can use it to do what i want.

May 8, 2012 at 1:48 PM

the JsonSerializeJazzBase() extension method you mentioned seems to serialize exactly what i want! From a DB point of view i can easily create a property on my object that returns the string from t hat method so it will get saved in the DB as well. 

Loading the state back onto the object seems harder or i am missing something.  I would have a new object that i have loaded and created from the DB table and i would want to use the json string generate from the method above to apply the state info to the object. IF this is already implemented i am unclear on how i am meant to do so.  DeserializeBaseJazz in the same class does not seem like the correct thing as i will already have the object and want to restore the state onto that object if at all possible.  I know you said you are still working on it so maybe it is not done yet but though i would check in case it is and i am missing something.

 

 

Coordinator
May 8, 2012 at 2:12 PM
seer_tenedos wrote:

the JsonSerializeJazzBase() extension method you mentioned seems to serialize exactly what i want! From a DB point of view i can easily create a property on my object that returns the string from t hat method so it will get saved in the DB as well. 

Loading the state back onto the object seems harder or i am missing something.  I would have a new object that i have loaded and created from the DB table and i would want to use the json string generate from the method above to apply the state info to the object. IF this is already implemented i am unclear on how i am meant to do so.  DeserializeBaseJazz in the same class does not seem like the correct thing as i will already have the object and want to restore the state onto that object if at all possible.  I know you said you are still working on it so maybe it is not done yet but though i would check in case it is and i am missing something.

 

 

I will break up the 'Deserializer' into 2 methods, "CreateBlankJazzObject' and 'Populate'.  The 'Populate' method  just populates the base fields an existing jazz object.  Reading your message I think that is what you need.

May 8, 2012 at 2:58 PM

that sounds perfect

May 8, 2012 at 10:07 PM

been playing some more and i am trying to work out how the following would work.

I have a WCF service that many different users can call. This service allows the users to perform many different tasks that allows them to load and manipulate objects.  That i think is quite simple as i just need to create a ClientNexus per request and load the object into it but maybe there is a better way?

 

I also have another WCF service for accessing and controlling the state on shared in memory long running tasks.  I was hoping to use Jazz to control access and state of these tasks but since they are in memory single instance tasks how can i have many users access the same instance at the same time. These tasks can run for up to 3-5 days downloading data with methods to start, stop, pause, resume, get status etc. My understanding is JObjects must be in a ClientNexus to be used for state and access control etc via roles but the can also only be in one ClientNexus at a time.

I am also a little worried about the overhead of creating a ClientNexus per request per user.  Using some sort of session my help with removing the objects out of the workspace after each request but i think that would only for for request on objects from storage not on long running in memory single instance objects where more than one user could access it at the same time.

My system uses something similar to WCF but the biggest difference i my system is designed to be split over many boxes and with live load balancing and potential live deployment of the service to new boxes based on load requirements. Just wanted to be clear that when i say WCF i mean WCF like but not WCF itself. It places a few restrictions on me and i have not worked out how those restrictions will effect Jazz yet. Very nice system called ICE by ZeroC if you are interested.

 

Coordinator
May 9, 2012 at 12:43 AM
Edited May 9, 2012 at 11:27 AM

Lots of questions.  I will try to answer the best I can.  If I miss some just ask again.

1.  jObjects, a least copies of jObjects, are designed to be in multiple client nexuses at a time.  This will become more clear when ACL is introduced.   To have multiple client nexuses a coordinating server is required to hand out objects based the logged in profiles/user and on the roles (ACL) of the jObjects themselves.  This is planned, not yet implemented.   This all hinges on serialization which we have just implemented.   Many of the base fields, such 'timestamp', 'status', 'storagestatus' are used to help in that activity.

2. Yes jObjects need to be in a client nexus to have roles to work.   The algorithm is to check each request on a jObject against the roles of client nexus's logged in user/profile.  As you can see the client Nexus is required to give context.

public static bool IsAccessible_Roles(jObject jazzObject, IEnumerable<string> roles)       
{           
return (roles.Intersect(jazzObject.ClientNexus.LoginRoles.Select(r => r.Name)).Count() > 0);  
 }

3.  The cost of creating a client nexus is quite low.  It is basically a collection.  The cost scales linearly as function of the number of jObjects loaded.  There some cost for each Jazz object type doing cross checking (checking the jObject's code via reflection) and creating a jClass (schema object).  Deserialization of jObjects is the biggest cost in creating a client nexus.

4. You can load jobjects in client nexus and change the logged in user.   You have probably seen the example '2.RoleTests.cs'.  There is currently no penalty for changing users on a client nexus.  With the introduction of ACL this will change as each object will need to have its ACL checked and set to accessible, read only or not accessible.  Still the overhead per jObject is quite low.

Hope this helps,

John

May 9, 2012 at 3:24 AM

Thanks John.

I get the answers above but for me answer 1 above will already exist when i get that method to de-serialize the jobject info that you are separating out for me as the object store etc is just my existing system.

 

My big issue is working with some single instance objects that are concurrently used by multiple users doing multiple request on the same object at the same time. I could quite easily have 2 different threads from 2 different requests trying to add these objects to 2 different client nexus's at the same time to make sure the method calls buy the user and the states are correct. 

I get that the clientNexus needs to be aware of the jobject but why does the jobject care about the clientnexus or the number of client nexus it is used in? My application is mostly concurrent requests and while edit operations have checking etc to make that 1 at a time read-only operations are generally concurrent acting on the same object.  

 

Chris

Coordinator
May 9, 2012 at 12:00 PM
Edited May 9, 2012 at 12:36 PM

I think on a fundametal level the answer to your question on why a physical jObject can only be in a single nexus is 2. above.

It is allowed, and expected in many applications, that a single jObject will be copied, through serialization, and exist in multiple client nexus's.

The basic Jazz model is that each client Nexus represents objects (Jazz objects and other types) for a single (user/profile).  It is designed to be a working cache of objects (data and code) for a user(profile).  Some jObjects may be accessible for one user, while being totally inaccessible for another user.  The accessibility of these objects can be dynamic.

The client nexus is on a per user basis to support atomic operations where changes may require modification of multiple objects, but they need to changed as a whole.  Also a client nexus and its objects may be discarded to support edit/cancel operations.

The model does expect multiple physical users, such as multiple web page 'anonymous' users accessing a single client nexus.  In this scenario the physical users are actually 1 logical user.

I apologize if I have laboured on some of these points.  I am writting for you and other readers of this discussion.

If you would like more direct answers please feel free to contact me on skype -"Hansen.John" (voice only, I have no camera).

regards,

John

May 9, 2012 at 9:46 PM

Hi John you are right a misread the code in 2. above and did not notice it hit nexusclient.  I thought nexus client was passing it all the info it needed to work it out itself.

I think my issue is that i am trying to misuse your system.  My requirements for workflow are similar in that i need to have access and flow controlled by state and user but differ in that my workflow items are really objects that are doing things and updating other items for days via downloading and processing data.  I can just serialise that running object and access in another nexus client just to get its status or have another user try to stop it or something. I still have other cases that will match to what your system is designed for where i have items i get from the db, do something with, change state and store again so it will still be useful.

I just need to find something for my other cases.