All,
I seem to be unable to deploy an application that runs perfectly on my local cluster to a cluster in Azure.
Strange thing is that I was able to deploy this application in the past, but seem unable to now. Upon deployment of the application it seems that my stateless services are first deployed without any problems, but my statefull services and actors seem to be unable to locate suitable nodes for replicas, as I seem to be receiving the following warning (through the service fabric explorer, when selecting the application):
Unhealthy event: SourceId='System.PLB', Property='ServiceReplicaUnplacedHealth_Primary_8f41b6fb-3c6f-4916-9f85-27da0171ae6d', HealthState='Warning', ConsiderWarningAsError=false.
The Load Balancer was unable to find a placement for one or more of the Service's Replicas:
fabric:/Namespace/ActorService Primary Partition 8f41b6fb-3c6f-4916-9f85-27da0171ae6d could not be placed, possibly, due to the following constraints and properties:
Placement Constraint: N/A
Depended Service: N/A
Constraint Elimination Sequence:
NodeBlockList eliminated 5 possible node(s) for placement -- 0/5 node(s) remain.
Upon further investigation, I also encountered the following error:
Error event: SourceId='System.Hosting', Property='CodePackageActivation:Code:EntryPoint'.
ServiceManifestName='ActorPkg', NodeName='_nt1vm_4',AggregatedHealthState='Error'.
When checking all log files in blob storage, as well as the traces, I also found the following log statements:
2016-5-12 12:54:15.176,Warning,1516,2528,Hosting.ProcessActivationManager,Application service 66933b89-78e9-4590-8971-d0cad139d956 with processId 3240 for parent process 310a2a579b75d9847ba5a43d6b365d7c terminated with exit code 2148734720
2016-5-12 12:54:15.176,Warning,1516,2528,Hosting._Nodes_ProcessUnexpectedTermination@310a2a579b75d9847ba5a43d6b365d7c,ServiceHostProcess: Process.exe for ApplicationId 66933b89-78e9-4590-8971-d0cad139d956 terminated unexpectedly with exit code 2148734720 on node id 310a2a579b75d9847ba5a43d6b365d7c
2016-5-12 12:54:15.176,Informational,1516,2528,Hosting.ProcessActivationManager,Sending ApplicationService host down notification for apphost 66933b89-78e9-4590-8971-d0cad139d956, node 310a2a579b75d9847ba5a43d6b365d7c
2016-5-12 12:54:15.176,Warning,2888,2556,Hosting.ApplicationHostManager@310a2a579b75d9847ba5a43d6b365d7c:131075269508464788,Processing termination of activated Application Host: HostId=66933b89-78e9-4590-8971-d0cad139d956, ExitCode=2148734720
2016-5-12 12:54:15.176,Warning,2888,2556,IPC.ServerFailedToRemoveClient,527e532390 client 66933b89-78e9-4590-8971-d0cad139d956 not found
2016-5-12 12:54:15.176,Informational,2888,2556,Hosting.ApplicationHostManager@310a2a579b75d9847ba5a43d6b365d7c:131075269508464788,RemoveActivationTableEntry: ErrorCode=S_OK, HostId=66933b89-78e9-4590-8971-d0cad139d956
2016-5-12 12:54:15.176,Informational,1516,2528,Transport.Enqueue,adb77036b0 enqueue ef74d99d-1a29-4c34-b6d1-1f59542a42d8:1926 False 319B @ qsize 0/0B
2016-5-12 12:54:15.176,Informational,2864,2556,RA.HostingProcessClosedEvent_Hosting@310a2a579b75d9847ba5a43d6b365d7c:131075269508464788,RA on node 310a2a579b75d9847ba5a43d6b365d7c:131075269508464788 performing AppHost down processing for apphost: 66933b89-78e9-4590-8971-d0cad139d956/
2016-5-12 12:54:15.176,Informational,2864,2556,RA.MultipleFTWorkBegin_MultipleFT@310a2a579b75d9847ba5a43d6b365d7c:131075269508464788,Multiple FT Work Begin [Activity: AppHostClosed 66933b89-78e9-4590-8971-d0cad139d956]. FTs 36
2016-5-12 12:54:15.176,Informational,2888,2556,Hosting.HostingHealthManager@310a2a579b75d9847ba5a43d6b365d7c:131075269508464788,ServicePackage ReportHealth: HealthReport(fabric:/Namespace+ActorPkg+310a2a579b75d9847ba5a43d6b365d7c instance=131075312250519674 System.Hosting CodePackageActivation:Code:EntryPoint Error ttl=Infinite sn=131075312551846737 There was an error during CodePackage activation.The service host terminated with exit code:2148734720 removeWhenExpired=false 2016-05-12 12:54:15.184 priority NotAssigned)
If anybody has any ideas or suggestions as to why my application can no longer be deployed on the cloud that would be greatly appreciated!
Kind regards
A single Service Fabric node type/scale set can not contain more than 100 nodes/VMs. To scale a cluster beyond 100 nodes, add additional node types.
A Service Fabric cluster is a network-connected set of virtual or physical machines into which your microservices are deployed and managed. A machine or VM that is part of a cluster is called a cluster node. Clusters can scale to thousands of nodes.
In response to The Muffin Man's comment, I've removed my answer from the question and placed it here (and accepted it as a solution):
After figuring out how to turn out additional diagnostic information. (Via cloud explorer, navigate to the Virtual Machine Scale Sets and then select the scale set. In my case 'nt1vm' and choose Update Diagnostics... via Actions and turn on all logging)
I came upon this line (in the WADWindowsEventLogsTable):
.NET Runtime version : 4.0.30319.34209 - This application could not be started.This application requires one of the following versions of the .NET Framework: .NETFramework,Version=v4.6.1Do you want to install this .NET Framework version now?
So I'm downgrading my application to use .NET 4.5.2.
UPDATE SOLVED: After downgrading to .NET 4.5.2 my application deployed perfectly. Thanks for this post: Hosting console application in public service fabric cluster to let me try this.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With