Transfer gigantic DataTables over WCF / .Net Remoting

Capacity improvements package for .NET Remoting / WCF This solution deals with transferring huge DataTables over WCF and .NET Remoting. Imagine an online casino, needing to clear transactions at end of day. The clearing service needs to collect transaction data from the database, populated by all Poker servers, and Blackjack servers. Now the Reporting service needs to receive a huge DataTable, consisting of all online gaming transactions. It is very likely that the transport will fail, because the DataTable is tremendous. When transporting large DataTables between a Server and a remote Client, there are several issues; that stem from .NET serialization. Serialization of a large DataTable is memory thirsty. A large enough DataTable will cause the client to get System.OutOfMemoryException, or System.InsufficientMemoryException. Those exceptions cannot be caught at the server side, as they occur in the innards of the framework code that deals with seriailization and transport. If the DataTable is really large, the framework will throw the server process all together, with brute force. There is no way around that. Another issue is throughput, which is not outstanding, and becomes noticeable when the table gets large. The solution at hand circumvents this problem by partitioning the DataTable to chunks and transferring the chunks in a multi-threaded fashion. In a nutshell, the Server returns an object to the client, through which the client can make concurrent calls back at the server. This approach enables an unlimited-sized table to be transferred between server and client. It also improves throughput by up to a staggering factor of 3.5 (WCF can transfer a table at 3,425 Kb/sec, whereas the current solution boosts throughput to 12,245 Kb/sec), due to the concurrent requests. (Acts like a web accelerator)

DevLib

DevLib.zip (30,4 MB)
DevLib is a highly reusable collection of components and core functionalities addressing common crosscutting concerns.

Octopus Management Foundation

omf.zip (45 kB)
Octopus is a class library for distributed management of remote processes, brain functioning as operations in any information system, is under the MIT license

PowerShell Remoting

PowerShell Remoting is a light-weighted server-client application which allow you securely connect to remote PowerShell host and run script interactivly.

ProxyFactory

The emergence of the WCF platform announces the unification of all communications technologies provided by the .NET platform in a common foundation and so their decline, WCF’s adoption curve will be slow enterprise wide. So it is still safe bet to develop a library to leverage .NET 2.0 generics and SchemaImporterExtension mechanisms to offer a similar developer experience.

Distributed Publish/Subscribe (Pub/Sub) Event System

pubsub.zip (21,2 MB)
The Web Solutions Platform (WSP) event system is a general-purpose distributed publish/subscribe event system designed for high performance and low latency. Publish/subscribe is a common communication pattern though there are many variations to solve different scenario requirements. The event system provides the developer with a class library paradigm for publishing and subscribing to events. The events are developer-defined .Net classes which sub-class from the base event. This makes the system both easy to use and very extensible. With one line in the config file, events can be persisted to a log file in a form ready to BCP into SQL. The latest performance results that I have achieved using a 50 property .Net object (event object) on my development machine are: * 40,000+ events / second to publish (includes serialization) with no listener * 23,000+ events / second to publish (includes serialization) while persisting at 8,000+ events / second (includes deserialization and serialization) * 20,000+ events / second to publish (includes serialization) and sending across machines at 4,000+ events / second with latency <1ms The event system is currently running on all the www.microsoft.com production servers. A real-time monitoring application uses the event system for communications and monitors 100% of the IIS requests. Within 3 seconds, all http requests across all servers and all datacenters are aggregated, displayed on a console, and able to instigate alerts. Tags: architecture, BizTalk, distributed systems, ESB, event systems, eventing, Events, messaging, metrics, monitoring, networking, pub/sub, publish/subscribe, remoting, routing, Web Services, webservices

RemotingLite

Light weight remoting framework for both Mono.NET and Microsoft .NET. RemotingLite is a light framework for writing distributed service oriented applications via remoting. It utilizes Reflection to create clientside proxies, and communication between clients and host is done via messages through TCP/IP. The framework borrows ideas from Windows Communication Foundation, and can run on both the Microsoft .NET and Mono .NET runtimes. It has been tested in a mixed environment with Microsoft .NET runtime v. 2.0 and 3.0, and Mono .NET runtime v. 1.2.6.0.

基于SOA模式的轻量级服务调用框架模型 (KJFramework.ServiceModel)

********************************************************************************** * * KJFramework.ServiceModel 基于SOA模式的轻量级服务框架模型 * ********************************************************************************** [说明:] KJFramework框架中内置了一套轻量级RPC服务调用框架,使用此框架,将使得用户不再关心底层协议转换和网络传输的繁琐过程,解放程序员的脑力劳动。 此框架的使用也非常简单,作为服务器端的使用人员,只需要完成一个契约的实现,就可以通过配置不同的底层通信信道来达到不同通信模式需求。 在客户端,我们可以由2种方法来完成对于服务器端契约的使用: 1. 直接饮用服务器端契约文件(dll) 2. 使用内部专门编写的VS插件(VS Add-in)来完成对于远程服务端契约的动态生成 当然,我们推荐使用第二种方式,因为使用VS插件可以帮助用户自动生成针对于每一个契约操作的异步版本。 目前此内置框架被初步定义在命名空间:KJFramework.ServiceModel内,拥有独立Assembly. 目前此框架中,仍保留了内部扩展能力,日后扩展后,将会通过配置文件来增强配置能力。

Service Invocation Framework

SIF.zip (961 kB)
Lightwave framework for service invocation. Includes pipelines for cross-cutting logic (transactions, logging, security), adapters for common transport protocols (MTOM, SOAP, Remoting, etc) and a dynamic proxy factory (ala WCF).

Softix.Orchestration

SimpleFlow.zip (1,8 MB)
A simple workflow engine written in .NET that will allow non-coders to create and run workflows (like automatic parsing of emails, call center scripts etc)

SPLaunch (PowerShell remoting automation)

splaunch.zip (499 kB)
Wrapper to automate PowerShell remoting. Two CSV files hold target server names and shortcut cmdlets so you can easily mix and match to run without typing much.

SQL Spatial Tile Render Engine

sstre.zip (39 kB)
CLR CSharp application to render a set of GIS spatial images directly from SQL Server as rowset data using standard T-SQL language extended for simplicity . C# 3.5 32 or 64 bit extension for SQL server 2008

Terminals

Terminals.zip (155,5 MB)
Terminals is a secure, multi tab terminal services/remote desktop client. It uses Terminal Services ActiveX Client (mstscax.dll).

THN Teleport

A .NET communication abstraction layer that enables developers to write the same code for different network topologies and technologies.

WCF vs Remoting

Project shows performance comparison between Windows Communication Foundation and .NET Remoting

#Zyan Communication Framework

zyan.zip (61,7 MB)
Zyan is an easy to use distributed application framework for .NET, Mono and Android. With Zyan you can publish any .NET class for remote access over network.