Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge hotfix/v2.0.2 into master #194

Merged
merged 45 commits into from
Feb 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
c74d954
fix Bug 65354
SuhorukovAnton Jan 19, 2024
f6f7c2a
ASC.Web.Core: Notify: fixed ArgumentNullException: Value cannot be nu…
andreysavihin Jan 23, 2024
31bc9f6
Merge pull request #168 from ONLYOFFICE/bugfix/fixed-notify-exception
pavelbannov Jan 23, 2024
cd682e9
ASC.Core.Common: Notify: additional logging
andreysavihin Jan 23, 2024
c4c9395
Merge pull request #169 from ONLYOFFICE/feature/notify-logs
pavelbannov Jan 23, 2024
815b739
rabbitmq: fixed
alexeybannov Jan 23, 2024
f59c921
rabbitmq: dll version update
alexeybannov Jan 23, 2024
e9b5206
ASC.Api.Core: InvitationLinkHelper: optimization
andreysavihin Jan 24, 2024
f0cb3f1
Merge pull request #170 from ONLYOFFICE/feature/confirm-optimization
pavelbannov Jan 24, 2024
54a3a4d
Arabic and Serbian empty file templates
LinneyS Jan 22, 2024
db5415a
Thumbnails: fixed generate
pavelbannov Jan 24, 2024
73fd5e8
fix bug 65943
pavelbannov Jan 24, 2024
8914d6f
fix backup/restore
SuhorukovAnton Jan 24, 2024
16977d2
fix Bug 66027
pavelbannov Jan 24, 2024
80b3a9f
ASC.Common: TimeZoneConverter: added GetIanaTimeZoneId method
andreysavihin Jan 24, 2024
fa6042e
Fix incorrect SsoUrl (AuthN request issue)
AlexeySafronov Jan 24, 2024
4a89565
Merge pull request #172 from ONLYOFFICE/bugfix/fixed-timezone-exception
pavelbannov Jan 25, 2024
6a6d5c4
Merge pull request #163 from ONLYOFFICE/fix/Bug-65354
pavelbannov Jan 25, 2024
c5fb47a
rabbitmq: log shutdown event
alexeybannov Jan 25, 2024
7486ea8
rabbitmq: fixed log
alexeybannov Jan 25, 2024
46f8698
fix bug 66001
SuhorukovAnton Jan 25, 2024
5b11462
Search: fix memory
pavelbannov Jan 25, 2024
339e56b
Merge pull request #173 from ONLYOFFICE/bugfix/66001
pavelbannov Jan 25, 2024
d75f82e
using options for rate limiter
alexeybannov Jan 25, 2024
ca4c2ec
ratelimiting: update dll
alexeybannov Jan 25, 2024
4b67470
Merge branch 'hotfix/v2.0.2' of github.com:ONLYOFFICE/DocSpace-server…
alexeybannov Jan 25, 2024
adb6fef
DbQuota: new quota
pavelbannov Jan 25, 2024
db5845a
filehandler: add range in stream
SuhorukovAnton Jan 26, 2024
e1b49e2
fix
SuhorukovAnton Jan 26, 2024
d457d6f
apisystem: if param tenantRegion is null, when will throw exception
alexeybannov Jan 29, 2024
60288f6
Merge pull request #176 from ONLYOFFICE/feature/filehandler-stream-range
pavelbannov Jan 29, 2024
c911f4f
revert db5845a8d76f704032a66bed2daba6c2c3cf702e
pavelbannov Jan 30, 2024
227b2f9
fixed support range header
pavelbannov Jan 30, 2024
59bcc4c
fixed Bug 66053
alexeybannov Feb 1, 2024
c1d7608
fixed Bug 65781
alexeybannov Feb 1, 2024
8ee3fdb
optimized query
MaksimChegulov Feb 1, 2024
14a7f75
support cors *
alexeybannov Feb 1, 2024
ae0155d
Merge pull request #182 from ONLYOFFICE/bugfix/slow-query
alexeybannov Feb 1, 2024
03e33b6
Elastic: fixed threads
pavelbannov Feb 1, 2024
5abae06
ASC.Web.Core: Notify: new welcome mail
andreysavihin Feb 2, 2024
dab8106
ASC.Files: DocumentService: Configuration: porting bug fix 66304
andreysavihin Feb 2, 2024
18e8b0d
Merge pull request #185 from ONLYOFFICE/feature/pdf-embed-logo
pavelbannov Feb 2, 2024
d9fc0dd
Merge pull request #184 from ONLYOFFICE/feature/zoom-welcome-mail
pavelbannov Feb 2, 2024
6c4cfab
ASC.Files: FilesCommonResource: oform replacement
andreysavihin Feb 2, 2024
6a11fee
Merge pull request #186 from ONLYOFFICE/feature/pdf-oform-replace
pavelbannov Feb 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions common/ASC.Api.Core/ASC.Api.Core.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
<PackageReference Include="Microsoft.Extensions.Hosting.Systemd" Version="7.0.0" />
<PackageReference Include="Microsoft.Extensions.Hosting.WindowsServices" Version="7.0.1" />
<PackageReference Include="NLog.Web.AspNetCore" Version="5.3.5" />
<PackageReference Include="RedisRateLimiting" Version="1.0.12" />
<PackageReference Include="RedisRateLimiting.AspNetCore" Version="1.0.9" />
<PackageReference Include="RedisRateLimiting" Version="1.1.0" />
<PackageReference Include="RedisRateLimiting.AspNetCore" Version="1.1.0" />
<PackageReference Include="StackExchange.Redis.Extensions.AspNetCore" Version="9.1.0" />
<PackageReference Include="StackExchange.Redis.Extensions.Newtonsoft" Version="9.1.0" />
<PackageReference Include="Autofac.Extensions.DependencyInjection" Version="8.0.0" />
Expand Down
83 changes: 74 additions & 9 deletions common/ASC.Api.Core/Core/BaseStartup.cs
Original file line number Diff line number Diff line change
Expand Up @@ -104,14 +104,50 @@ public virtual void ConfigureServices(IServiceCollection services)

services.AddRateLimiter(options =>
{
bool EnableNoLimiter(IPAddress address)
{
var knownNetworks = _configuration.GetSection("core:hosting:rateLimiterOptions:knownNetworks").Get<List<String>>();
var knownIPAddresses = _configuration.GetSection("core:hosting:rateLimiterOptions:knownIPAddresses").Get<List<String>>();

if (knownIPAddresses is { Count: > 0 })
{
foreach (var knownIPAddress in knownIPAddresses)
{
if (IPAddress.Parse(knownIPAddress).Equals(address)) return true;
}
}

if (knownNetworks is { Count: > 0 })
{
foreach (var knownNetwork in knownNetworks)
{
var prefix = IPAddress.Parse(knownNetwork.Split("/")[0]);
var prefixLength = Convert.ToInt32(knownNetwork.Split("/")[1]);
var ipNetwork = new IPNetwork(prefix, prefixLength);

if (ipNetwork.Contains(address)) return true;
}
}

return false;
}


options.GlobalLimiter = PartitionedRateLimiter.CreateChained(
PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
{
var userId = httpContext?.User.Claims.FirstOrDefault(x => x.Type == ClaimTypes.Sid)?.Value;

var remoteIpAddress = httpContext?.Connection.RemoteIpAddress;

if (EnableNoLimiter(remoteIpAddress))
{
return RateLimitPartition.GetNoLimiter("no_limiter");
}

if (userId == null)
{
userId = httpContext?.Connection.RemoteIpAddress.ToInvariantString();
userId = remoteIpAddress.ToInvariantString();
}

var permitLimit = 1500;
Expand All @@ -131,9 +167,16 @@ public virtual void ConfigureServices(IServiceCollection services)
string partitionKey;
int permitLimit;

var remoteIpAddress = httpContext?.Connection.RemoteIpAddress;

if (EnableNoLimiter(remoteIpAddress))
{
return RateLimitPartition.GetNoLimiter("no_limiter");
}

if (userId == null)
{
userId = httpContext?.Connection.RemoteIpAddress.ToInvariantString();
userId = remoteIpAddress.ToInvariantString();
}

if (String.Compare(httpContext?.Request.Method, "GET", StringComparison.OrdinalIgnoreCase) == 0)
Expand All @@ -154,14 +197,21 @@ public virtual void ConfigureServices(IServiceCollection services)
QueueLimit = 0,
ConnectionMultiplexerFactory = () => connectionMultiplexer
});
}),
}),
PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
{
var userId = httpContext?.User?.Claims.FirstOrDefault(x => x.Type == ClaimTypes.Sid)?.Value;

var remoteIpAddress = httpContext?.Connection.RemoteIpAddress;

if (EnableNoLimiter(remoteIpAddress))
{
return RateLimitPartition.GetNoLimiter("no_limiter");
}

if (userId == null)
{
userId = httpContext?.Connection.RemoteIpAddress.ToInvariantString();
userId = remoteIpAddress.ToInvariantString();
}

var partitionKey = $"fw_post_put_{userId}";
Expand All @@ -187,6 +237,12 @@ public virtual void ConfigureServices(IServiceCollection services)
var userId = httpContext?.User?.Claims.FirstOrDefault(x => x.Type == ClaimTypes.Sid)?.Value;
var permitLimit = 5;
var partitionKey = $"sensitive_api_{userId}";
var remoteIpAddress = httpContext?.Connection.RemoteIpAddress;

if (EnableNoLimiter(remoteIpAddress))
{
return RateLimitPartition.GetNoLimiter("no_limiter");
}

return RedisRateLimitPartition.GetSlidingWindowRateLimiter(partitionKey, _ => new RedisSlidingWindowRateLimiterOptions
{
Expand Down Expand Up @@ -254,10 +310,14 @@ public virtual void ConfigureServices(IServiceCollection services)
policy =>
{
policy.WithOrigins(_corsOrigin)
.SetIsOriginAllowedToAllowWildcardSubdomains()
.AllowAnyHeader()
.AllowAnyMethod()
.AllowCredentials();
.SetIsOriginAllowedToAllowWildcardSubdomains()
.AllowAnyHeader()
.AllowAnyMethod();

if (_corsOrigin != "*")
{
policy.AllowCredentials();
}
});
});
}
Expand Down Expand Up @@ -411,7 +471,12 @@ public virtual void Configure(IApplicationBuilder app, IWebHostEnvironment env)
app.UseAuthentication();

// TODO: if some client requests very slow, this line will need to remove
// app.UseRateLimiter();
bool.TryParse(_configuration["core:hosting:rateLimiterOptions:enable"], out var enableRateLimiter);

if (enableRateLimiter)
{
app.UseRateLimiter();
}

app.UseAuthorization();

Expand Down
13 changes: 7 additions & 6 deletions common/ASC.Api.Core/Security/InvitationLinkHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ private async Task<ValidationResult> ValidateIndividualLinkAsync(string email, s
return ValidationResult.Invalid;
}

var visitMessage = await GetLinkVisitMessageAsync(email, key);
var visitMessage = await GetLinkVisitMessageAsync(user.TenantId, email, key);

if (visitMessage == null)
{
Expand All @@ -151,14 +151,14 @@ private async Task<ValidationResult> ValidateIndividualLinkAsync(string email, s
return linkId == default ? (ValidationResult.Invalid, default) : (ValidationResult.Ok, linkId);
}

private async Task<DbAuditEvent> GetLinkVisitMessageAsync(string email, string key)
private async Task<DbAuditEvent> GetLinkVisitMessageAsync(int tenantId, string email, string key)
{
await using var context = await _dbContextFactory.CreateDbContextAsync();

var target = _messageTarget.Create(email);
var description = JsonSerializer.Serialize(new[] { key });

var message = await Queries.AuditEventsAsync(context, target.ToString(), description);
var message = await Queries.AuditEventsAsync(context, tenantId, target.ToString(), description);

return message;
}
Expand Down Expand Up @@ -188,8 +188,9 @@ public class LinkValidationResult

static file class Queries
{
public static readonly Func<MessagesContext, string, string, Task<DbAuditEvent>> AuditEventsAsync =
public static readonly Func<MessagesContext, int, string, string, Task<DbAuditEvent>> AuditEventsAsync =
EF.CompileAsyncQuery(
(MessagesContext ctx, string target, string description) =>
ctx.AuditEvents.FirstOrDefault(a => a.Target == target && a.DescriptionRaw == description));
(MessagesContext ctx, int tenantId, string target, string description) =>
ctx.AuditEvents.FirstOrDefault(a =>
a.TenantId == tenantId && a.Action == (int)MessageAction.RoomInviteLinkUsed && a.Target == target && a.DescriptionRaw == description));
}
2 changes: 1 addition & 1 deletion common/ASC.Common/ASC.Common.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
<PackageReference Include="NLog.Targets.Syslog" Version="7.0.0" />
<PackageReference Include="NVelocity" Version="1.2.0" />
<PackageReference Include="protobuf-net" Version="3.2.26" />
<PackageReference Include="RabbitMQ.Client" Version="6.6.0" />
<PackageReference Include="RabbitMQ.Client" Version="6.8.1" />
<PackageReference Include="StackExchange.Redis" Version="2.6.122" />
<PackageReference Include="StackExchange.Redis.Extensions.Core" Version="9.1.0" />
</ItemGroup>
Expand Down
12 changes: 12 additions & 0 deletions common/ASC.Common/Utils/TimeZoneConverter/TimeZoneConverter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,18 @@ public string GetTimeZoneDisplayName(TimeZoneInfo tz)
return displayName;
}

public string GetIanaTimeZoneId(TimeZoneInfo timeZone)
{
if (timeZone.HasIanaId)
{
return timeZone.Id;
}

return TimeZoneInfo.TryConvertWindowsIdToIanaId(timeZone.Id, out var ianaTimeZoneId)
? ianaTimeZoneId
: WindowsTzId2OlsonTzId(timeZone.Id);
}

public string OlsonTzId2WindowsTzId(string olsonTimeZoneId, bool defaultIfNoMatch = true)
{
var mapZone = GetMapZoneByWindowsTzId(olsonTimeZoneId);
Expand Down
11 changes: 10 additions & 1 deletion common/ASC.Core.Common/EF/Model/Tenant/DbQuota.cs
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,18 @@ public static ModelBuilderWrapper AddDbQuota(this ModelBuilderWrapper modelBuild
Price = 0,
ProductId = null,
Visible = false
},
new DbQuota
{
TenantId = -8,
Name = "zoom",
Description = null,
Features = "free,total_size:107374182400,manager:100,room:100",
Price = 0,
ProductId = null,
Visible = false
}
);

return modelBuilder;
}

Expand Down
4 changes: 2 additions & 2 deletions common/ASC.Core.Common/Log/NotifyEngineLogger.cs
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ public static partial class NotifyEngineLogger
[LoggerMessage(Level = LogLevel.Error, Message = "NotifySender")]
public static partial void ErrorNotifySender(this ILogger<NotifySenderService> logger, Exception exception);

[LoggerMessage(Level = LogLevel.Error, Message = "Prepare")]
public static partial void ErrorPrepare(this ILogger logger, Exception exception);
[LoggerMessage(Level = LogLevel.Error, Message = "Prepare {action} {recipient}")]
public static partial void ErrorPrepare(this ILogger logger, Exception exception, INotifyAction action, IRecipient recipient);

[LoggerMessage(Level = LogLevel.Error, Message = "UpdateScheduleDate")]
public static partial void ErrorUpdateScheduleDate(this ILogger logger, Exception exception);
Expand Down
2 changes: 1 addition & 1 deletion common/ASC.Core.Common/Notify/Engine/NotifyEngine.cs
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ private async Task<List<SendResponse>> SendDirectNotifyAsync(NotifyRequest reque
catch (Exception ex)
{
responses.Add(new SendResponse(request.NotifyAction, null, request.Recipient, SendResult.Impossible));
_logger.ErrorPrepare(ex);
_logger.ErrorPrepare(ex, request.NotifyAction, request.Recipient);
}

if (request._senderNames is { Length: > 0 })
Expand Down
27 changes: 11 additions & 16 deletions common/ASC.Data.Backup.Core/BackupAjaxHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,8 @@ public async Task CheckAccessToFolderAsync<T>(T folderId)
public async Task CreateScheduleAsync(BackupStorageType storageType, Dictionary<string, string> storageParams, int backupsStored, CronParams cronParams)
{
await DemandPermissionsBackupAsync();

await DemandPermissionsAutoBackupAsync();

if (!SetupInfo.IsVisibleSettings("AutoBackup"))
{
throw new InvalidOperationException(Resource.ErrorNotAllowedOption);
Expand Down Expand Up @@ -238,7 +239,7 @@ public async Task<Schedule> GetScheduleAsync()
schedule.StorageParams = consumer.AdditionalKeys.ToDictionary(r => r, r => consumer[r]);
schedule.StorageParams.Add("module", "S3");

var Schedule = new CreateScheduleRequest
var scheduleRequest = new CreateScheduleRequest
{
TenantId = await _tenantManager.GetCurrentTenantIdAsync(),
Cron = schedule.CronParams.ToString(),
Expand All @@ -247,7 +248,7 @@ public async Task<Schedule> GetScheduleAsync()
StorageParams = schedule.StorageParams
};

await _backupService.CreateScheduleAsync(Schedule);
await _backupService.CreateScheduleAsync(scheduleRequest);

}
else if (response.StorageType != BackupStorageType.ThirdPartyConsumer)
Expand Down Expand Up @@ -314,37 +315,31 @@ public async Task StartRestoreAsync(string backupId, BackupStorageType storageTy
public async Task<BackupProgress> GetRestoreProgressAsync()
{
var tenant = await _tenantManager.GetCurrentTenantAsync();
var result = _backupService.GetRestoreProgress(tenant.Id);

return result;

return _backupService.GetRestoreProgress(tenant.Id);
}

public async Task DemandPermissionsRestoreAsync()
private async Task DemandPermissionsRestoreAsync()
{
await _permissionContext.DemandPermissionsAsync(SecurityConstants.EditPortalSettings);

var quota = await _tenantManager.GetTenantQuotaAsync(await _tenantManager.GetCurrentTenantIdAsync());
if (!SetupInfo.IsVisibleSettings("Restore") ||
(!_coreBaseSettings.Standalone && !quota.AutoBackupRestore))
if (!SetupInfo.IsVisibleSettings("Restore") || (!_coreBaseSettings.Standalone && !quota.AutoBackupRestore))
{
throw new BillingException(Resource.ErrorNotAllowedOption, "Restore");
}


if (!_coreBaseSettings.Standalone
&& (!SetupInfo.IsVisibleSettings("Restore")
|| !quota.AutoBackupRestore))
if (!_coreBaseSettings.Standalone && (!SetupInfo.IsVisibleSettings("Restore") || !quota.AutoBackupRestore))
{
throw new BillingException(Resource.ErrorNotAllowedOption, "Restore");
}
}

public async Task DemandPermissionsAutoBackupAsync()
private async Task DemandPermissionsAutoBackupAsync()
{
await _permissionContext.DemandPermissionsAsync(SecurityConstants.EditPortalSettings);

if (!SetupInfo.IsVisibleSettings("AutoBackup") ||
(!_coreBaseSettings.Standalone && !(await _tenantManager.GetTenantQuotaAsync(await _tenantManager.GetCurrentTenantIdAsync())).AutoBackupRestore))
if (!SetupInfo.IsVisibleSettings("AutoBackup") || !(await _tenantManager.GetTenantQuotaAsync(await _tenantManager.GetCurrentTenantIdAsync())).AutoBackupRestore)
{
throw new BillingException(Resource.ErrorNotAllowedOption, "AutoBackup");
}
Expand Down
30 changes: 4 additions & 26 deletions common/ASC.Data.Backup.Core/Tasks/BackupPortalTask.cs
Original file line number Diff line number Diff line change
Expand Up @@ -123,26 +123,6 @@ private async Task DoDump(IDataWriteOperator writer)
{
var databases = new Dictionary<Tuple<string, string>, List<string>>();

try
{
await using var connection = DbFactory.OpenConnection();
var command = connection.CreateCommand();
command.CommandText = "select id, connection_string from mail_server_server";
ExecuteList(command).ForEach(r =>
{
var connectionString = GetConnectionString((int)r[0], JsonConvert.DeserializeObject<Dictionary<string, object>>(Convert.ToString(r[1]))["DbConnection"].ToString());

var command = connection.CreateCommand();
command.CommandText = "show tables";
var tables = ExecuteList(command).Select(r => Convert.ToString(r[0])).ToList();
databases.Add(new Tuple<string, string>(connectionString.Name, connectionString.ConnectionString), tables);
});
}
catch (Exception e)
{
_logger.ErrorWithException(e);
}

await using (var connection = DbFactory.OpenConnection())
{
var command = connection.CreateCommand();
Expand Down Expand Up @@ -470,26 +450,24 @@ private void SaveToFile(string path, string t, IReadOnlyCollection<string> colum

for (var i = 0; i < obj.Length; i++)
{
var value = obj[i];
if (value is byte[] byteArray && byteArray.Length != 0)
var byteArray = obj[i] as byte[];
if (byteArray != null && byteArray.Length != 0)
{
sw.Write("0x");
foreach (var b in byteArray)
{
sw.Write("{0:x2}", b);
}
}
else
{
var s = obj[i] as string;
if (s != null)
{
sw.Write("'" + s.Replace("\r", "\\r").Replace("\n", "\\n") + "'");
sw.Write("'" + s.Replace("\\", "\\\\").Replace("\r", "\\r").Replace("'", "\\'").Replace("\n", "\\n") + "'");
}
else
{
var ser = new JsonSerializer();
ser.Serialize(writer, value);
ser.Serialize(writer, obj[i]);
}
}
if (i != obj.Length - 1)
Expand Down
Loading
Loading